-- Logs begin at Wed 2022-09-07 18:23:29 UTC, end at Wed 2022-09-07 19:35:49 UTC. -- Sep 07 18:54:49.732066 dv-cinder-847732-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 18:54:49.741815 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 18:54:49 2022] *** Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: nodename: dv-cinder-847732-5-iscsi Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: machine: x86_64 Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: clock source: unix Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: pcre jit disabled Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: detected number of CPU cores: 4 Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: current working directory: / Sep 07 18:54:49.744479 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: detected binary path: /usr/bin/uwsgi-core Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: your processes number limit is 47680 Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: your memory page size is 4096 bytes Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: detected max file descriptor number: 2048 Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: lock engine: pthread robust mutexes Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: thunder lock: enabled Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 18:54:49.745129 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 18:54:49.784712 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: Python main interpreter initialized at 0x55ce82bd99c0 Sep 07 18:54:49.784712 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: python threads support enabled Sep 07 18:54:49.784712 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: your server socket listen backlog is limited to 100 connections Sep 07 18:54:49.784712 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: your mercy for graceful operations on workers is 90 seconds Sep 07 18:54:49.785033 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 18:54:49.785108 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: *** Operational MODE: preforking *** Sep 07 18:54:49.786008 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: *** uWSGI is running in multiple interpreter mode *** Sep 07 18:54:49.786008 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: spawned uWSGI master process (pid: 81589) Sep 07 18:54:49.786214 dv-cinder-847732-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 18:54:49.788182 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: spawned uWSGI worker 1 (pid: 81601, cores: 1) Sep 07 18:54:49.788182 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: spawned uWSGI worker 2 (pid: 81602, cores: 1) Sep 07 18:54:49.788182 dv-cinder-847732-5-iscsi devstack@keystone.service[81589]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 18:54:54.114411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 18:54:54.114411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Sep 07 18:54:54.115241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 18:54:54.115241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Sep 07 18:54:54.115347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 18:54:54.115347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: from cryptography.utils import int_from_bytes Sep 07 18:54:54.115347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 18:54:54.115347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: from cryptography.utils import int_from_bytes Sep 07 18:54:54.165227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 18:54:54.165403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 18:54:54.165403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 18:54:54.165584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 18:54:54.165673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] command line args: [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 18:54:54.165753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] command line args: [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 18:54:54.165861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 18:54:54.165942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 18:54:54.166020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ================================================================================ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 18:54:54.166159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ================================================================================ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 18:54:54.166279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] admin_token = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.166530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config_dir = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.166651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.166727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.166912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config_source = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] debug = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_dir = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_source = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.167971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] debug = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_config_append = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.168952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_dir = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_config_append = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.169930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_dir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.170884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.171955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.172082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.172197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.172321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.172437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.172603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.172723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] publish_errors = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.173963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.174082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.174217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.174439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.174576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.174694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] standard_threads = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.174923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.175048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.175179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.176651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.176779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.176974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_journal = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.176974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_json = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.176974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_stderr = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.177302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.177428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_syslog = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.177550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] publish_errors = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.177670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.177785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.177904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.178018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.178018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.178245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.178365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.178480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.178624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.178754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.178877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] standard_threads = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.178992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.179117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.179232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.179351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.179478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.179594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.179713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.179827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_journal = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.179940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.180059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_json = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.180246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.180246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.180462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_stderr = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.180576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.180688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_syslog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.180801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.180801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 18:54:54.181141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.181818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.182998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.183947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.184900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.185898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.186973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.187088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.187213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.187213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.187693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.187815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.187929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.188971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.189881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.190957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.191061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.191172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.191324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.191961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.192900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.193920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.194987 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.195930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.196967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.197984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.198999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.199842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.200984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.201921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.202994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.external = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.password = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.203930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.token = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.204960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.205938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.206910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.207984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.external = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.208776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.209391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.209505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.209611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.209611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.209799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.209896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.password = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.token = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.210940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.211917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.212923 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.213932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.214984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.215922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.216953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.217887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.218960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.219880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.220901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.221939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.222911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.223944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.224883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.225933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.226998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.227920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.228964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.229888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.230946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.231917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.232970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.233932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.234954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.235898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.236955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.237911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.238028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.238655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.238776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.238891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239584 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.239940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.240980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.241905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.242964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.243907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.244902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.245928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.246886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.247957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.248883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.249938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.250910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.251973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.252094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.252094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 18:54:54.252312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.252312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.252535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.252650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.252903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.253138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.253383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.253651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.254453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.254724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.254975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.255215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.255459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.255696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.255934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.256172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.256410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.256656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.256894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.257140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.257378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.257626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.257864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.258110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.258342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.258608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.258849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 18:54:54.259020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 18:54:54.281646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.281878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.282106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.283828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.284059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.284282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.291253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.293365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.319598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.319856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.321528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.321758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.373235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.373659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.422024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.422255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.422652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.422887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.578690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.578915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.579867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.580009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.603927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.604011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.604146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.604227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.604369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.604451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.604558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.616027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.616296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.628615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.628830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.629984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.630108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.630321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 18:54:54.647688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.647897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 18:54:54.648235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.648378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.648461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 18:54:54.648589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 18:54:54.648696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.648928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.649018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.649131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 18:54:54.649212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.649407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.649488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.649671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.649928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.650186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.659041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.659321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.659589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.659866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.659950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.660694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.660969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.661254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.667712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.668503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.670620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.670910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.671195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.671413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.671496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.671702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.671785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.671989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.672879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.673063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.673143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.673327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.673406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.673598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.673875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.674145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.696035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.696825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.697485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.697770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.698284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.698587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.702437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.702747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.703033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.703336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.703626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.703909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.709890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.710177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.710878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.711170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.715800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.716101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.716392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.716682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.716764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.716965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.717877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.718148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.718422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.718717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.733767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.734045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.734668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.734951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.737869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.738778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.741103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.741399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.741957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.742254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.745580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.746412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.748731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.749536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.751985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.752285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.752573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.752756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.752855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.753056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.753138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.753339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.753420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.753623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.753948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.754239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.769197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.770007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.770278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.770503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.770604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.770827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.771098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.771364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.777387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.778207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.778949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.779775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.780719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.781033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.781287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.781573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.783984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.784278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.784410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.784565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.784697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.784849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.785044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.785792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.785893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.786203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.797225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.797503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.797628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.797783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.797917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.798062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.798203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.798448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.798538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.798755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.798887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.799052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.799194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.799478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.814963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.815068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.817936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.818106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.820831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.821013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.822270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.822449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.823762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.824052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.824452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.824634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.825038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.825444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.830858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.832844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.834274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.835768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.835855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.836169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.836459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.836739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.839902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.840301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.840590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.840871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.844543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.844847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.845129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.849374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.849666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.849948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.852647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.852933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.853217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.853495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.853782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.854057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.854332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.854641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.854927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.855207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.855524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.855834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.856151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.856443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.857519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 18:54:54.857807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.858088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.858365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.858667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.858942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.859232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.859651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.860055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.860425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.860711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.860978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.861246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.861534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 18:54:54.890424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ce82bd99c0 pid: 81601 (default app) Sep 07 18:54:54.893700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be45b2-8d08-4f4e-820c-f8290e1eedca None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:54:54.893985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be45b2-8d08-4f4e-820c-f8290e1eedca None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:54:54.894258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be45b2-8d08-4f4e-820c-f8290e1eedca None None] PATH_INFO: `/v3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:54:54.896840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ce82bd99c0 pid: 81602 (default app) Sep 07 18:54:54.898992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Wed Sep 7 18:54:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:02.425002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:02.425531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:02.425611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:02.429937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:02.435336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:02.435770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:02.435867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:02.506673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: INFO dbcounter [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] Registered counter for database keystone Sep 07 18:55:02.525740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG oslo_db.sqlalchemy.engines [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 18:55:02.531936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writer thread running {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 18:55:02.711624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG passlib.handlers.bcrypt [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81601) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 18:55:02.712011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG passlib.handlers.bcrypt [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81601) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 18:55:02.769579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:02.771161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:02.897510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 07 18:55:03.367167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:03.437807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: INFO dbcounter [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] Registered counter for database keystone Sep 07 18:55:03.459083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG oslo_db.sqlalchemy.engines [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 18:55:03.465480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writer thread running {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 18:55:03.587880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc1625-0caa-4f0c-8078-a85294b9e27b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:03.592165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:03.592789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:03.593071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:03.593344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:03.602125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:03.654443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING keystone.common.rbac_enforcer.enforcer [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 18:55:03.654866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d0935-964b-4589-8c9b-bb850fb7ee67 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:03.665117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Wed Sep 7 18:55:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 18:55:03.671082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:03.687894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dd0fe-6a6e-4160-89a9-131d06e2f477 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:03.692284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:03.693393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:03.693678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:03.693959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] PATH_INFO: `/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:03.695586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] RBAC: Authorizing `identity:get_project(project_id=0eeffadf972f483fb4e32d2c25b5e94c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:03.748775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 18:55:03.749123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605c8bc-3854-4cb4-b5f7-fe40424499cd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:03.753719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Wed Sep 7 18:55:03 2022] GET /identity/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:05.851132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c654467-8127-493e-9f11-47735fa64fd2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:05.851672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c654467-8127-493e-9f11-47735fa64fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:05.851672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c654467-8127-493e-9f11-47735fa64fd2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:05.855865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:05.861494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:05.861859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:05.862047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:05.882341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:05.883907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:05.891727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 18:55:06.160165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c654467-8127-493e-9f11-47735fa64fd2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:06.172499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c654467-8127-493e-9f11-47735fa64fd2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:06.177304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf2e29-a39e-4aad-a2f2-df7a994c8bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:06.177899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf2e29-a39e-4aad-a2f2-df7a994c8bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:06.178188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf2e29-a39e-4aad-a2f2-df7a994c8bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:06.178461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf2e29-a39e-4aad-a2f2-df7a994c8bc6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:06.186139 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf2e29-a39e-4aad-a2f2-df7a994c8bc6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:06.187943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf2e29-a39e-4aad-a2f2-df7a994c8bc6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:06.197255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Wed Sep 7 18:55:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 18:55:06.202640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:06.211810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23082d94-62a4-4e2f-9c47-e588c4a0de0e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:06.216169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2558c-2573-447b-9467-0247f51e5f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:06.216795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2558c-2573-447b-9467-0247f51e5f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:06.217078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2558c-2573-447b-9467-0247f51e5f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:06.217351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2558c-2573-447b-9467-0247f51e5f84 admin admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:06.218697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2558c-2573-447b-9467-0247f51e5f84 admin admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:06.220528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2558c-2573-447b-9467-0247f51e5f84 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:06.222089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Wed Sep 7 18:55:06 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:08.453472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01459012-f752-4c0e-b4b4-9bb9349f3d4e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:08.454003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01459012-f752-4c0e-b4b4-9bb9349f3d4e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:08.454130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01459012-f752-4c0e-b4b4-9bb9349f3d4e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:08.455514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:08.461205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:08.461599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:08.461760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:08.466227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df68a-bfc4-41a2-a791-bc2c67aecbb3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:08.466593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df68a-bfc4-41a2-a791-bc2c67aecbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:08.466876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df68a-bfc4-41a2-a791-bc2c67aecbb3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:08.468097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:55:08.473693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:08.473973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:08.474246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:08.481112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:08.482279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:08.515456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81602) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 18:55:08.515774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81602) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 18:55:08.565113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:08.565962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:08.567270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:08.665238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:08.834216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:08.857914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:08.927457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:08.937349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a344b-d0b1-46d4-aa82-a5d774a30576 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:08.972106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:08.972743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:08.973024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:08.973310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:08.984475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:08.993855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:08.994895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:08.995665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:09.000721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:09.026575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543e544-e08c-4ac2-beb3-7651c20aaa71 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.028897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:09.064575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.066085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.066362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.066682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.086231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:09.103316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d230d3-7be5-4c8d-aaa8-fb4737961acc None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.134493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.136019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.137983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.138953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.139230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.139498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.140165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:09.140465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.141833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.153709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:09.159487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:09.241857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:55:09.247354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:09.249998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:09.263904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5785cec-feed-4947-8fca-1f14d8b15246 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.297799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.298431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.298739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.299017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.300704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.302157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.303772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:09 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:09.309102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:09.323069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:09.345764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52849920-ff86-420b-8f8c-b4c529e16e37 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.380210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.381052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.381325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.381592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.399777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.401239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.406807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549c50b-b879-42e6-bbc2-da46805f0282 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:55:09.419917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:09.420783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:55:09.421611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:55:09.421611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:55:09.421611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:09.421611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 9/18] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:55:09.427402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:09.442702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.443340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.443624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.443901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.445049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.446616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.448247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 10/19] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:09.453697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:09.484441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:09.530351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3519f-dda8-4445-835a-ccf101f269db None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.533347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:09.552760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20d4cf-58aa-415c-b1d0-ca04bcba127e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:09.564824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.565460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.565880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.566154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.566833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:09.567135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.568525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f9636-05e6-49a2-9b00-dfbb8c46cd44 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.587427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 10/20] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:09.589026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:09.589642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:09.589918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:09.590188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:09.590960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.domain.id=default, user.id=bc4a17a3aa754cf9b61c2b367071735b) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:09.591271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:09.592751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaacf3e-2169-43b0-871e-7e72cc97c1be None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:09.601147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 11/21] 10.222.0.46 () {56 vars in 1322 bytes} [Wed Sep 7 18:55:09 2022] GET /identity/v3/role_assignments?user.id=bc4a17a3aa754cf9b61c2b367071735b&scope.domain.id=default&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:12.063675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47150be0-b28e-49ec-a11c-32587719a361 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.064261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47150be0-b28e-49ec-a11c-32587719a361 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.064261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47150be0-b28e-49ec-a11c-32587719a361 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.072011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 11/22] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:12.077526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225addc-c67b-4540-8dae-c8b3787f07d8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.078051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225addc-c67b-4540-8dae-c8b3787f07d8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.078051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225addc-c67b-4540-8dae-c8b3787f07d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.104562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6225addc-c67b-4540-8dae-c8b3787f07d8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:12.105866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6225addc-c67b-4540-8dae-c8b3787f07d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:12.192985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 12/23] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:12.218851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee510bb2-e42c-44e8-be15-8694dd903fb5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.219148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee510bb2-e42c-44e8-be15-8694dd903fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.219406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee510bb2-e42c-44e8-be15-8694dd903fb5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.220801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 12/24] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:12.226145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffed09a-dd98-45b1-9b36-a4863470794b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.231603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffed09a-dd98-45b1-9b36-a4863470794b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.231603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffed09a-dd98-45b1-9b36-a4863470794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.257651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3ffed09a-dd98-45b1-9b36-a4863470794b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:12.264363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3ffed09a-dd98-45b1-9b36-a4863470794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:12.349654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 13/25] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:12.571346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a832f5-96b0-44ab-be76-4d7e17521221 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.571727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a832f5-96b0-44ab-be76-4d7e17521221 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.571909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a832f5-96b0-44ab-be76-4d7e17521221 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.573541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 13/26] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:12.583933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.584218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.584494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.597797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:12.608898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:12.610156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:12.620010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a832f5-96b0-44ab-be76-4d7e17521221 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:12.740433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 14/27] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:12.764532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:12.820307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a832f5-96b0-44ab-be76-4d7e17521221 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:12.879019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:12.886065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:12.886730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:12.887007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:12.887281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:12.901126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:12.910767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:55:12.939051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:12.940610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:55:12.941478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:55:12.941478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:55:12.941478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:12.941478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 14/28] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:12.947744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:12.996010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4da459-21d3-4238-b842-245de42f3317 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:13.075938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:13.076607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.076893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.077170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.080497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:13.115376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:13.120712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:13.127358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:13.128667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:13.129305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c9793-7938-4039-9e15-5fe2d1716ac2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:13.129720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 15/29] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:13.131398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:13.199379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:13.200003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.200288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.200563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.201223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:13.201528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:13.202939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:13.228764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 15/30] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:13.238448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:13.285464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:13.293363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a420c-7b85-449a-acf2-4dc6979aa776 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:13.331339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:13.332234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.332465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.332807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.333590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:13.333908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:13.335417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:13.344327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 16/31] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:13.350448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:13.380420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bc598-5418-42ee-972a-5b79a52bd798 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:13.420643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:13.429745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a100a7a-7c75-4d4e-8c15-1010972bcfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:13.432074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a100a7a-7c75-4d4e-8c15-1010972bcfce None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.432074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a100a7a-7c75-4d4e-8c15-1010972bcfce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.432074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a100a7a-7c75-4d4e-8c15-1010972bcfce None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.432805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a100a7a-7c75-4d4e-8c15-1010972bcfce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:13.435465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a100a7a-7c75-4d4e-8c15-1010972bcfce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:13.472154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ed5df-746b-4d90-9bd8-1f2459c1aaed None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:13.508882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbeaa0e-f349-4dbc-b2fd-32c500c7a6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:13.509502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeaa0e-f349-4dbc-b2fd-32c500c7a6b0 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.509820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeaa0e-f349-4dbc-b2fd-32c500c7a6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.510161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeaa0e-f349-4dbc-b2fd-32c500c7a6b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.510850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbeaa0e-f349-4dbc-b2fd-32c500c7a6b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:13.512369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbeaa0e-f349-4dbc-b2fd-32c500c7a6b0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:13.649550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 16/32] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Sep 7 18:55:13 2022] POST /identity/v3/roles => generated 219 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:13.651416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042772c1-1f89-4b23-93ee-2f9d8ae56053 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.651924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042772c1-1f89-4b23-93ee-2f9d8ae56053 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.652402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042772c1-1f89-4b23-93ee-2f9d8ae56053 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.653909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 17/33] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:13.654907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042772c1-1f89-4b23-93ee-2f9d8ae56053 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:13.732712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 17/34] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:55:13 2022] POST /identity/v3/projects => generated 301 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:13.734139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46c499-d6d3-4874-82a5-c2a5514fa48a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.734540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46c499-d6d3-4874-82a5-c2a5514fa48a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.734860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46c499-d6d3-4874-82a5-c2a5514fa48a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.736108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 18/35] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:13.737782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.738064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.738341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.760132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:13.761983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:13.770846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:13.796486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042772c1-1f89-4b23-93ee-2f9d8ae56053 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:13.842091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d09a3-0a67-4de3-a646-90ef29083b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:13.842896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d09a3-0a67-4de3-a646-90ef29083b8d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.843242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d09a3-0a67-4de3-a646-90ef29083b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.843585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d09a3-0a67-4de3-a646-90ef29083b8d None admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.845776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d09a3-0a67-4de3-a646-90ef29083b8d None admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:13.848797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d09a3-0a67-4de3-a646-90ef29083b8d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:13.850954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 18/36] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:13 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:13.852021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf529-c408-4927-83bc-92ec94bd5f49 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:13.852376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf529-c408-4927-83bc-92ec94bd5f49 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:13.852728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf529-c408-4927-83bc-92ec94bd5f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:13.881588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cddaf529-c408-4927-83bc-92ec94bd5f49 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:13.883856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cddaf529-c408-4927-83bc-92ec94bd5f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:13.906682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 19/37] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:13.906682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:13.956337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:14.042209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265985fa-f48c-4450-85fb-2e69d9dad2a7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:14.046683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 19/38] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:14.048799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069469b0-868a-44f7-9919-28b3373100b9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.049099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069469b0-868a-44f7-9919-28b3373100b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.049417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069469b0-868a-44f7-9919-28b3373100b9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.050736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 20/39] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:14.059079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.060393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.060393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.087382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:14.088630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:14.089981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:14.090816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.091104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.091391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.092688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:14.094502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:14.096654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 20/40] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:14.102863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:14.178320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 21/41] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:14.262775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:14.282284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:14.347621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689d4b1-67b5-461c-b520-28d6e16ecfd7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:14.356811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:14.380597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67edb9-1fe7-4185-80d9-a582cdde437b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:14.389208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:14.389878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.390165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.390586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] PATH_INFO: `/v3/domains/default/users/bc4a17a3aa754cf9b61c2b367071735b/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.392641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bc4a17a3aa754cf9b61c2b367071735b, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:14.394743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:14.417654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:14.418264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.418564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.418845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.419517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:14.420966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:14.489366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 21/42] 10.222.0.46 () {58 vars in 1361 bytes} [Wed Sep 7 18:55:14 2022] PUT /identity/v3/domains/default/users/bc4a17a3aa754cf9b61c2b367071735b/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 18:55:14.490108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:14.521641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 22/43] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Sep 7 18:55:14 2022] POST /identity/v3/roles => generated 225 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:55:14.527958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:14.615630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a1411-0159-431c-94c4-5bbf4f024493 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:14.659812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:14.661188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf9fd8-c677-4f37-962b-41cdfbc9a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:14.662094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9fd8-c677-4f37-962b-41cdfbc9a474 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.662618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9fd8-c677-4f37-962b-41cdfbc9a474 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.663100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9fd8-c677-4f37-962b-41cdfbc9a474 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.664290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf9fd8-c677-4f37-962b-41cdfbc9a474 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:14.666017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf9fd8-c677-4f37-962b-41cdfbc9a474 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:14.690599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214ca6-8cd5-4a1c-a22b-bfd003f6dc58 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:14.727436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:14.728321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.728603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.728879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.730008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:14.731494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:14.733177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 23/44] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:14.739773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:14.766938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 22/45] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Sep 7 18:55:14 2022] POST /identity/v3/roles => generated 223 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:14.857018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b93fb5-1831-49fe-8035-412e69df5831 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:14.894427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684a352-d2c8-4901-b0a0-149c94f49feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:14.895100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684a352-d2c8-4901-b0a0-149c94f49feb None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:14.895377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684a352-d2c8-4901-b0a0-149c94f49feb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:14.895648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684a352-d2c8-4901-b0a0-149c94f49feb None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:14.896334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684a352-d2c8-4901-b0a0-149c94f49feb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:14.897821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684a352-d2c8-4901-b0a0-149c94f49feb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:14.917620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:15.030808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 24/46] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:55:14 2022] POST /identity/v3/projects => generated 312 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:55:16.666512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2975b-4a48-4a8d-b034-1eff4db6e22f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:16.667086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2975b-4a48-4a8d-b034-1eff4db6e22f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:16.667086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2975b-4a48-4a8d-b034-1eff4db6e22f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:16.668478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 23/47] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:16.673936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:16.674939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:16.675031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:16.693357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:16.694542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:16.759312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 25/48] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:17.027044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2975b-4a48-4a8d-b034-1eff4db6e22f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.055833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:17.135503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2975b-4a48-4a8d-b034-1eff4db6e22f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.169855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f78399-5a14-4995-b4ec-1cf0a687f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.170490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f78399-5a14-4995-b4ec-1cf0a687f573 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.170800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f78399-5a14-4995-b4ec-1cf0a687f573 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.171076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f78399-5a14-4995-b4ec-1cf0a687f573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.181082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f78399-5a14-4995-b4ec-1cf0a687f573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-09f78399-5a14-4995-b4ec-1cf0a687f573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:17.186399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:17.187247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:17.188565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 24/49] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:17.193543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.203673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f5b9-9000-4751-99d1-2a455eb29e07 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.203956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f5b9-9000-4751-99d1-2a455eb29e07 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.204234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f5b9-9000-4751-99d1-2a455eb29e07 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.206049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 25/50] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:55:17.211040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.211321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.211594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.229575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:17.230777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:17.295948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f80a97-7bf6-42ed-915e-87fce84ead05 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.297043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 26/51] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:17.321690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:17.330006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.336387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.336387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.336387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.336387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:17.336387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:17.336387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:17.345650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 26/52] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:17.351786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.369188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:17.450461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b95be-5e9b-4e86-bd35-da540e7e2fd6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.483708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.484331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.484632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.484904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.486018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:17.487541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:17.489107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 27/53] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:17 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:17.494657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.565522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.596154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa905c-9f0f-4557-b0c1-7364a261eb6b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.629504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.630133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.630415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.630722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.631835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:17.633285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:17.634896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 27/54] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:17.640557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.649742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:17.665889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacfa92-f7e8-4e8c-845e-369c7fad7a24 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.698220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:17.699803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.700438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.700736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.700997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.702118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:17.703603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:17.705244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 28/55] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:17.710822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:17.743244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab61bf-a1c0-4561-a8fc-6fe453b1f8fd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.815675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd920f2f-b129-41fd-a06a-1ac467df1894 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:17.850978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51224d00-fec4-4c3e-923d-5d957fd61882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.851592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224d00-fec4-4c3e-923d-5d957fd61882 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.851871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224d00-fec4-4c3e-923d-5d957fd61882 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.852144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224d00-fec4-4c3e-923d-5d957fd61882 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.852803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51224d00-fec4-4c3e-923d-5d957fd61882 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:17.854258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51224d00-fec4-4c3e-923d-5d957fd61882 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:17.880289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:17.880887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:17.881166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:17.881501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:17.882168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.domain.id=default, user.id=bc4a17a3aa754cf9b61c2b367071735b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:17.882478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:17.883927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925ceaa-b9ee-4a76-b501-c76e44777cf1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:17.892368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 28/56] 10.222.0.46 () {56 vars in 1322 bytes} [Wed Sep 7 18:55:17 2022] GET /identity/v3/role_assignments?user.id=bc4a17a3aa754cf9b61c2b367071735b&scope.domain.id=default&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 530 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:17.949114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 29/57] 10.222.0.46 () {60 vars in 1140 bytes} [Wed Sep 7 18:55:17 2022] POST /identity/v3/projects => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:55:20.117885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a72ae-f419-4eb7-9d44-bdb62523d3bc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:20.118425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a72ae-f419-4eb7-9d44-bdb62523d3bc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:20.118425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a72ae-f419-4eb7-9d44-bdb62523d3bc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:20.119990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 29/58] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:20.125081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:20.125476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:20.125629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:20.132920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:20.144240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:20.145418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:20.213337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 30/59] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:20.481686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a72ae-f419-4eb7-9d44-bdb62523d3bc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:20.579755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a72ae-f419-4eb7-9d44-bdb62523d3bc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:20.599291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:20.612509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de983e1-49da-4149-9671-474390add2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:20.613122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de983e1-49da-4149-9671-474390add2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:20.613399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de983e1-49da-4149-9671-474390add2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:20.613672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de983e1-49da-4149-9671-474390add2f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:20.614851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de983e1-49da-4149-9671-474390add2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:20.616281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de983e1-49da-4149-9671-474390add2f1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:20.617812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 30/60] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:20.623277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:20.726371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d92bb1-8c33-4a46-a051-9ebbc215ee98 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:20.759556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebe3a0-0ab1-455c-9db0-034577e7ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:20.760103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebe3a0-0ab1-455c-9db0-034577e7ad91 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:20.760387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebe3a0-0ab1-455c-9db0-034577e7ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:20.760662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebe3a0-0ab1-455c-9db0-034577e7ad91 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:20.761311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebe3a0-0ab1-455c-9db0-034577e7ad91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:20.762783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebe3a0-0ab1-455c-9db0-034577e7ad91 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:20.773080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:20.860570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 31/61] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:20 2022] POST /identity/v3/users => generated 275 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:24.009651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e552e-846d-4c8c-923a-d346bce0502e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.010524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e552e-846d-4c8c-923a-d346bce0502e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.010969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e552e-846d-4c8c-923a-d346bce0502e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.012725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 31/62] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:24.018150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765152-dbb7-4195-b451-36ba1bd4801d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.018881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765152-dbb7-4195-b451-36ba1bd4801d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.019316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765152-dbb7-4195-b451-36ba1bd4801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.043495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-81765152-dbb7-4195-b451-36ba1bd4801d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:24.044893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-81765152-dbb7-4195-b451-36ba1bd4801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:24.131359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 32/63] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:24.251264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db07a5-06d9-41c4-8960-fa5468673e02 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.252373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db07a5-06d9-41c4-8960-fa5468673e02 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.252828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db07a5-06d9-41c4-8960-fa5468673e02 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.254351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 32/64] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:24.266960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.266960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.266960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.287557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:24.289006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:24.378404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 33/65] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:24.588939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db07a5-06d9-41c4-8960-fa5468673e02 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:24.724275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db07a5-06d9-41c4-8960-fa5468673e02 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:24.753159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:24.773608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafeabd-558a-46bb-bb1b-4a8bb52c0d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:24.774275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafeabd-558a-46bb-bb1b-4a8bb52c0d40 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.774577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafeabd-558a-46bb-bb1b-4a8bb52c0d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.774853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafeabd-558a-46bb-bb1b-4a8bb52c0d40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.789610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafeabd-558a-46bb-bb1b-4a8bb52c0d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-aaafeabd-558a-46bb-bb1b-4a8bb52c0d40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:24.796824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:24.797768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:24.799145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 33/66] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:24.800509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad88e-a625-42a3-9073-da81ca4b6d41 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.800509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad88e-a625-42a3-9073-da81ca4b6d41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.805168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad88e-a625-42a3-9073-da81ca4b6d41 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.806816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 34/67] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:24.807628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad88e-a625-42a3-9073-da81ca4b6d41 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:24.818318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:24.874142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:24.875928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6f784-2ed6-49b6-8436-e59516d824c3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:24.912001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c67938-e567-4c33-b542-f067ef694a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:24.912638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c67938-e567-4c33-b542-f067ef694a55 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.912922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c67938-e567-4c33-b542-f067ef694a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.913199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c67938-e567-4c33-b542-f067ef694a55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.924130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c67938-e567-4c33-b542-f067ef694a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-42c67938-e567-4c33-b542-f067ef694a55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:24.929030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:24.929877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:24.930669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:24.931263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad88e-a625-42a3-9073-da81ca4b6d41 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:24.931336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 34/68] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:55:24.931894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.932184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.932465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.949958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:24.951189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:24.964535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:24.965142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.965428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.965699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.966432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:24.966768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:24.968144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b032b-2c45-4667-91b4-991e9a166620 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:24.976403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 35/69] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:24.977554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2821e-ff9a-492d-bbfd-dfa0e883b0e4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.977991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2821e-ff9a-492d-bbfd-dfa0e883b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.978274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2821e-ff9a-492d-bbfd-dfa0e883b0e4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.979516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 36/70] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:24.983576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbda5d-2c48-40cb-9388-be5e8d7681d7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:24.983576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbda5d-2c48-40cb-9388-be5e8d7681d7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:24.983576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbda5d-2c48-40cb-9388-be5e8d7681d7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:24.983576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 37/71] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:24.983576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbda5d-2c48-40cb-9388-be5e8d7681d7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.026785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 35/72] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:25.028244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.075516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.101404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbda5d-2c48-40cb-9388-be5e8d7681d7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.114119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.137698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.138371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.138687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.138967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.139634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:25.139965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:25.141039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ef9d-6e14-4c7a-9f8c-bc4684d5e13c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.141346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d9c7a-e61f-423c-b16f-ddec64736387 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:25.151066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 38/73] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:25.152433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.153405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.154108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.173986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:25.175239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:25.180521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb33a0-2cdf-4ea5-b013-d0cb25612f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.181160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb33a0-2cdf-4ea5-b013-d0cb25612f01 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.181439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb33a0-2cdf-4ea5-b013-d0cb25612f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.181716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb33a0-2cdf-4ea5-b013-d0cb25612f01 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.196955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb33a0-2cdf-4ea5-b013-d0cb25612f01 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:25.198727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb33a0-2cdf-4ea5-b013-d0cb25612f01 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:25.200448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 36/74] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:25.201660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.201942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.202216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.220476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:25.221666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:25.247628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 39/75] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:25.258719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.307975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 37/76] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:25.308926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.338588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.371913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a96196-eaa4-4364-a2e0-287a0ede8b76 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.373571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.413320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.414533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.415118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.415400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.416697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:25.418250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:25.420219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 40/77] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:25.420993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.441425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52a1f0-309f-4cb5-976f-6bb5edac5dcc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.484193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.484965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.485248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.485805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.554063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbe646-a392-4e44-b77c-29127c06bea7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.572092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:25.573565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:25.596545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.597179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.597461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.597741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.601362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.608144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:25.613560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:25.614893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 41/78] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:25.615682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.649957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.677985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 38/79] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:25.678850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.728839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e4c82-ec44-4c5c-8f52-93152d8b166f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.762476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.763127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.763410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.763686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.778782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e72981-73af-4454-9bac-7022c4a8ee34 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.806191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.813570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.814238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.814507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.814807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.815912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:25.817355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:25.818962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 39/80] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:25.819752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.834864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:25.836370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:25.865822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:25.918595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5cf0f-25d6-4897-8eaa-4d830ebd82e6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:25.941177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 42/81] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:25.941948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:25.956334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:25.956964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:25.957255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:25.957531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:25.968123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:25.974112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:25.975006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:25.975926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 40/82] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:25.977189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.036431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.047455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a5a8-5a3e-436d-acfd-9ccd3a5bf259 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.077135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.080941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766ec49-86b4-4e6b-9509-81b9447cbd1c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.083494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.088233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.088233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.088233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.088233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:26.088233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.088233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.095294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 43/83] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.096073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.117680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.118317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.118622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.118903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.120052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.121524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.123201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 41/84] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:25 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.123965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.197630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c26c3e-5fc3-4bb0-9d4b-d732ef0f7500 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.227432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d7a6e-0643-4068-a924-5e1b85e4c0ab None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.234313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.234958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.235237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.235510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.236679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.238116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.250929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.261892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.266025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.266025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.266025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.266025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.266025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.267346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 42/85] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.268192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.340536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 44/86] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:26 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:26.343551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.372003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae8d76-0873-47ba-a327-ae68b3f09d66 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.400546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.416987 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.417614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.417896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.418175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.418890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:26.419259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.420687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.430788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 43/87] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.430788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.459047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4cfb5-6054-4c3b-aa1e-72a5d303036b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.500140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.501098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.501385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.502221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.502649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.503841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.505331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.507001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 45/88] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.512596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.556660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fceee-7bc3-4eea-9d49-c897a32c7be7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.593828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.594428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.594785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.595067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.596331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=87769f448fc54978888c2c57ac8ab26a) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:26.596674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.598214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.604080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 44/89] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/role_assignments?user.id=87769f448fc54978888c2c57ac8ab26a&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=8a09ae9946bc4e158d491383af34a89f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.604895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.644298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcc8c8-6deb-427a-888b-ffb9c355f9d9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.672392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.694653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.695467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.695758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.696034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.697389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=bc4a17a3aa754cf9b61c2b367071735b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:26.697702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.699590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.706756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 46/90] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/role_assignments?user.id=bc4a17a3aa754cf9b61c2b367071735b&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.708700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.763904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e6ea-db85-4d20-a0f7-291a78bdd738 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.790195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.823267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.823933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.824214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.824489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.828075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.832542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:26.835954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 45/91] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:26.842163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:26.856226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561e592-56f1-4819-a9ee-88ab16f7b87b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.901988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:26.902941 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:26.903247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:26.903745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:26.968669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:26.973136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75592-efad-47bf-829f-301f4ab70316 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:26.991335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:26.992966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:27.018349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:27.018917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:27.019200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:27.019474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:27.118939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:27.140721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:27.143838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:27.204609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 47/92] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:27.223656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:27.365462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 46/93] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:27.375591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:27.382103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:27.421800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db206c-79a9-40e3-b025-ead198ef5205 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:27.491112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:27.491112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:27.491112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:27.491112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:27.493580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:27.495232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:27.497097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 48/94] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:27 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:27.507422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:27.565871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:27.599796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc791-b9a9-4271-8653-525995d77b2a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:27.675604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de98c-c01e-4553-bce3-11d94901194a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:27.678038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:27.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:27.678999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:27.679277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] PATH_INFO: `/v3/projects/897c283a15da4e8eb49018829aa5c06f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:27.683377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] RBAC: Authorizing `identity:get_project(project_id=897c283a15da4e8eb49018829aa5c06f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:27.684866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:27.686564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 47/95] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:27 2022] GET /identity/v3/projects/897c283a15da4e8eb49018829aa5c06f => generated 312 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:27.693310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:27.747530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:27.748532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:27.749014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:27.749544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:27.750510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] RBAC: Adding query filter params (role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=87769f448fc54978888c2c57ac8ab26a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:27.751055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:27.753310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855b07e-6b43-432c-96c5-19cfc64516f3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:27.770942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 49/96] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:27 2022] GET /identity/v3/role_assignments?user.id=87769f448fc54978888c2c57ac8ab26a&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:27.885046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd55bd-d10b-4bd5-8fed-0ba88c0b544b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:27.937861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:27.938402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:27.938731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:27.939028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:27.939910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=897c283a15da4e8eb49018829aa5c06f, user.id=87769f448fc54978888c2c57ac8ab26a) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:27.940275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:27.941912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511d67-4d52-4fcc-adfb-d90f3a95f697 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:27.949518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 48/97] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:27 2022] GET /identity/v3/role_assignments?user.id=87769f448fc54978888c2c57ac8ab26a&scope.project.id=897c283a15da4e8eb49018829aa5c06f&role.id=8a09ae9946bc4e158d491383af34a89f => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:29.528088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4584-d30e-4062-bc2e-ac0def6c9c61 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:29.529097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4584-d30e-4062-bc2e-ac0def6c9c61 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:29.529585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4584-d30e-4062-bc2e-ac0def6c9c61 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:29.531610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 50/98] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:29.538870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0927caa-3644-4e32-a22f-9e8cb332c236 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:29.539846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0927caa-3644-4e32-a22f-9e8cb332c236 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:29.540326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0927caa-3644-4e32-a22f-9e8cb332c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:29.560217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e0927caa-3644-4e32-a22f-9e8cb332c236 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:29.561787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e0927caa-3644-4e32-a22f-9e8cb332c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:29.584849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:29.648960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 49/99] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:29.665425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12190332-c165-450b-b083-4e183db4f0e8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:29.665714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12190332-c165-450b-b083-4e183db4f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:29.665993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12190332-c165-450b-b083-4e183db4f0e8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:29.667386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 51/100] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:29.672815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7994-41ad-46b2-9622-80a159e0a0d4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:29.673761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7994-41ad-46b2-9622-80a159e0a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:29.674022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7994-41ad-46b2-9622-80a159e0a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:29.694816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fcdf7994-41ad-46b2-9622-80a159e0a0d4 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:29.696083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fcdf7994-41ad-46b2-9622-80a159e0a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:29.786250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 50/101] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:29.920529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19076-3f89-4372-89cf-112b3fab7bb3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:29.921149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19076-3f89-4372-89cf-112b3fab7bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:29.921624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19076-3f89-4372-89cf-112b3fab7bb3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:29.923742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 52/102] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:29.929298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:29.929931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:29.930406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:29.949307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:29.950802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:30.020278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 51/103] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:30.165831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a19076-3f89-4372-89cf-112b3fab7bb3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.184713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:30.227132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.244647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:30.352266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a19076-3f89-4372-89cf-112b3fab7bb3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:30.389162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:30.389802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.390082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.390356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.402387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:30.408506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:30.409367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:30.410138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 53/104] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:30.416562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.418073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff3d1a-3df4-4c7a-b674-ae3d56d0208e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:30.490779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:30.491431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.491713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.491990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.492514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:30.493168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:30.502774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:30.504468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 52/105] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:30.505330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.528441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e2084-2e7f-449d-8089-7cb2f12815ac None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:30.575298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:30.576008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.576288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.576602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.578936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:30.579252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:30.582485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:30.602701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 54/106] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:30.604556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.692883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:30.742762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162999-735d-411a-80d6-26f8a0ad7fb8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:30.766267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2d991-eef4-4ecc-aa92-7761000f3742 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:30.780671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:30.781951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.782432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.783149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.797495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:30.804124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:30.805172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:30.806110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:30.807115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f1be6-d8f2-4d77-8b08-7e789b57914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:30.807825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f1be6-d8f2-4d77-8b08-7e789b57914a None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.808200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f1be6-d8f2-4d77-8b08-7e789b57914a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.808687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f1be6-d8f2-4d77-8b08-7e789b57914a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.809106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 53/107] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:30.809416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f1be6-d8f2-4d77-8b08-7e789b57914a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:30.810300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.811164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f1be6-d8f2-4d77-8b08-7e789b57914a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:30.816859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:30.900775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 55/108] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:55:30 2022] POST /identity/v3/projects => generated 302 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:30.900775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe5db-4167-4c6c-894a-2a3c7546b484 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.900775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe5db-4167-4c6c-894a-2a3c7546b484 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.900775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe5db-4167-4c6c-894a-2a3c7546b484 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.900775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 56/109] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:30.904864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe5db-4167-4c6c-894a-2a3c7546b484 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:30.927980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762b1ff-0f30-4814-a244-be444037bddf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:30.974662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdea47-de35-4fe0-81ef-001fb83ebec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:30.975350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdea47-de35-4fe0-81ef-001fb83ebec1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.976126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdea47-de35-4fe0-81ef-001fb83ebec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.976427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdea47-de35-4fe0-81ef-001fb83ebec1 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.977942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdea47-de35-4fe0-81ef-001fb83ebec1 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:30.979948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdea47-de35-4fe0-81ef-001fb83ebec1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:30.982075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 54/110] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:30 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:30.983519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfc6e-7016-47ac-ae3a-22e7a53026ee None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.983806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfc6e-7016-47ac-ae3a-22e7a53026ee None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.984133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfc6e-7016-47ac-ae3a-22e7a53026ee None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:30.990738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 55/111] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:30.992281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:30.992621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:30.992902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.003189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:31.015598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:31.016921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:31.057402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe5db-4167-4c6c-894a-2a3c7546b484 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.099117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 56/112] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:31.100320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.109225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.110747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.111041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.111393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.112153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:31.112498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:31.114652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74dd4ac-d131-4a49-82df-5b1fe6e391eb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:31.131431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 57/113] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:31.136025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634a26-c64f-4fe9-8e69-4cc55d97f42f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.136244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634a26-c64f-4fe9-8e69-4cc55d97f42f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.136774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634a26-c64f-4fe9-8e69-4cc55d97f42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.155467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a6634a26-c64f-4fe9-8e69-4cc55d97f42f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:31.156680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6634a26-c64f-4fe9-8e69-4cc55d97f42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:31.216098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:31.233762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c75f95-c887-430f-a195-1b36e2e6704a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.248380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 58/114] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:31.254569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95fa3-a4b7-4571-98bc-f93dc464ea7b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.255319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95fa3-a4b7-4571-98bc-f93dc464ea7b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.255752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95fa3-a4b7-4571-98bc-f93dc464ea7b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.257697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 59/115] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:31.258608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95fa3-a4b7-4571-98bc-f93dc464ea7b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.286246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112d7d4-90f5-46ef-a84f-bb893ef43524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.288645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112d7d4-90f5-46ef-a84f-bb893ef43524 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.288937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112d7d4-90f5-46ef-a84f-bb893ef43524 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.289216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112d7d4-90f5-46ef-a84f-bb893ef43524 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.292844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112d7d4-90f5-46ef-a84f-bb893ef43524 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:31.294325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112d7d4-90f5-46ef-a84f-bb893ef43524 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:31.301742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 57/116] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:31 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:31.302945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.303228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.303502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.335363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:31.346573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:31.347762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:31.414418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95fa3-a4b7-4571-98bc-f93dc464ea7b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.439983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 58/117] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:31.441078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.465929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.466723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.467016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.467301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] PATH_INFO: `/v3/users/4dc813aca93149f799bbc6c108d84521` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.486104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] RBAC: Authorizing `identity:get_user(user_id=4dc813aca93149f799bbc6c108d84521)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:31.487780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:31.489350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 60/118] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:31 2022] GET /identity/v3/users/4dc813aca93149f799bbc6c108d84521 => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:31.490133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.546920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:31.595951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64f95a-c86b-433f-8895-af8fbfe2ff1b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.632216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a12a8a-104d-4361-ae46-11b9925fdf14 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.634816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:31.641564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.643060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.644110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d/users/87769f448fc54978888c2c57ac8ab26a/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.647576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=20b1e025ed514544abb2932a8dace90d, user_id=87769f448fc54978888c2c57ac8ab26a, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:31.649941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:31.670875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.671604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.671951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.672299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.683305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:31.688288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:31.693136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 61/119] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:31.693914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.727862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 59/120] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:31 2022] PUT /identity/v3/projects/20b1e025ed514544abb2932a8dace90d/users/87769f448fc54978888c2c57ac8ab26a/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:31.732528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.815164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:31.822444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d09fe5-a423-417d-9299-f3f6794f81ab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.863195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.864151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.864575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.865025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.875808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f19ff-e997-45c6-a534-6a41f58c72b7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:31.922427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:31.923659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:31.923941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:31.924212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:31.937769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:31.943647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:31.944658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:31.945518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:31.947200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:31.947870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 60/121] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:31.948852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:31.948988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.012967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.058982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.068415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 62/122] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:32.069417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.069591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6f1ef-c015-46fd-8a8e-4ef63baaea4f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.110262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.111182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.111601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.112017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.112822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:32.113296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.115009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.123810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 61/123] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.124884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.196666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6bf5e-bcc9-4164-81c4-4fd08814cd8b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.234985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.236636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.237166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.237635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.249800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:32.253244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2903f-a79b-4441-85bf-a4ec0967f73c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:32.255894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:32.256765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:32.257527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:32.259896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 63/124] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:32.260843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.268926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.291272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.292051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.292432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.292806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.293644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:32.294081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.295828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.305837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 62/125] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.306975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.332514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.382892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e294f48-2683-45d7-a1cf-cc7c4ab85f29 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.421386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.422254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.422716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.423159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.424446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.426056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.427876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 64/126] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.428942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.443660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe163a4-b2a8-4636-899d-3e2ff1f10c00 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.484007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.484805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.485187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.486003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.487629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.489927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.491978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 63/127] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.493171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.553079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.555501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7dea3-198e-49f1-a930-2a6b5bf71e21 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.589079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.603242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.604374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.604476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.604989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.605796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:32.606457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.608719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.618180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 65/128] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.623604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.623723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b8c76-31c1-430a-8b48-f0ce0f6e1f60 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.669404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.670064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.670395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.670728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.672634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.675192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.678040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 64/129] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.679065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.737109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325ad1-2bab-4d23-8c58-8670aec08bac None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.780076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.781056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.781557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.782030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.784103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] RBAC: Adding query filter params (role.id=f448f4dca91e4f1982f116ee3ff052b6, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=4dc813aca93149f799bbc6c108d84521) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:32.784624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.786911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.795042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 66/130] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/role_assignments?user.id=4dc813aca93149f799bbc6c108d84521&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=f448f4dca91e4f1982f116ee3ff052b6 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.796061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:32.836213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51f205-09e5-4ccc-8e36-2a1a0c5a7cf4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:32.874227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.923666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:32.925275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:32.926380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:32.926967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:32.929135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:32.934212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:32.936157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:32.939568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 65/131] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:32.942457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:33.004823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e969c99-2846-4850-b2b4-0dd336a49de8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:33.065553 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:33.066646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.067536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.068260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.071215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:33.074645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:33.080677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 67/132] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:33.081693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:33.129592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31e8cd-c745-4075-8806-0e812690e1ea None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:33.196018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:33.196669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.196950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.197227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] PATH_INFO: `/v3/projects/897c283a15da4e8eb49018829aa5c06f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.200349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] RBAC: Authorizing `identity:get_project(project_id=897c283a15da4e8eb49018829aa5c06f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:33.201880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:33.205653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 66/133] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:32 2022] GET /identity/v3/projects/897c283a15da4e8eb49018829aa5c06f => generated 312 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:33.206521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:33.248548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195481-fab6-4bc4-a689-9b7295cbc290 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:33.267546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:33.296163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:33.311356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e5e7e-21e5-4af2-ac33-110147fb658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:33.312022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5e7e-21e5-4af2-ac33-110147fb658a None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.312302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5e7e-21e5-4af2-ac33-110147fb658a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.312582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5e7e-21e5-4af2-ac33-110147fb658a None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d/users/bc4a17a3aa754cf9b61c2b367071735b/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.315321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e5e7e-21e5-4af2-ac33-110147fb658a None admin] RBAC: Authorizing `identity:create_grant(project_id=20b1e025ed514544abb2932a8dace90d, user_id=bc4a17a3aa754cf9b61c2b367071735b, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:33.317883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e5e7e-21e5-4af2-ac33-110147fb658a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:33.369864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8a6fa-cbeb-47c5-be2b-53e1a87a1b63 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:33.408295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:33.408847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.409127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.409465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.410623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:33.412074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:33.414266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 67/134] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:33 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:33.415520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:33.426610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 68/135] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:33 2022] PUT /identity/v3/projects/20b1e025ed514544abb2932a8dace90d/users/bc4a17a3aa754cf9b61c2b367071735b/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:33.430625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8938d7-0313-4fec-9abb-5bbd606418eb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.430625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8938d7-0313-4fec-9abb-5bbd606418eb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.430625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8938d7-0313-4fec-9abb-5bbd606418eb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.430625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 69/136] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:33.435472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8938d7-0313-4fec-9abb-5bbd606418eb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:33.589045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f2c5a-0dee-42bc-aa80-587037194321 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:33.616165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:33.640402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef97eb-a346-45e0-b35b-71a1fb05fffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:33.641095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97eb-a346-45e0-b35b-71a1fb05fffc None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.645373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97eb-a346-45e0-b35b-71a1fb05fffc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.645373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97eb-a346-45e0-b35b-71a1fb05fffc None admin] PATH_INFO: `/v3/projects/897c283a15da4e8eb49018829aa5c06f/users/87769f448fc54978888c2c57ac8ab26a/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.645530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8938d7-0313-4fec-9abb-5bbd606418eb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:33.645929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef97eb-a346-45e0-b35b-71a1fb05fffc None admin] RBAC: Authorizing `identity:create_grant(project_id=897c283a15da4e8eb49018829aa5c06f, user_id=87769f448fc54978888c2c57ac8ab26a, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:33.646188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef97eb-a346-45e0-b35b-71a1fb05fffc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:33.681761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50a682-f5d4-4a2e-8101-2004343e7278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:33.682413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50a682-f5d4-4a2e-8101-2004343e7278 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.682763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50a682-f5d4-4a2e-8101-2004343e7278 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.683046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50a682-f5d4-4a2e-8101-2004343e7278 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d/users/87769f448fc54978888c2c57ac8ab26a/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.685337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50a682-f5d4-4a2e-8101-2004343e7278 None admin] RBAC: Authorizing `identity:create_grant(project_id=20b1e025ed514544abb2932a8dace90d, user_id=87769f448fc54978888c2c57ac8ab26a, role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:33.686823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50a682-f5d4-4a2e-8101-2004343e7278 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:33.723301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 68/137] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:33 2022] PUT /identity/v3/projects/897c283a15da4e8eb49018829aa5c06f/users/87769f448fc54978888c2c57ac8ab26a/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:33.724548 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017a940-1438-46eb-8a03-b9aa6c8bd402 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:33.724836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017a940-1438-46eb-8a03-b9aa6c8bd402 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:33.725109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017a940-1438-46eb-8a03-b9aa6c8bd402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:33.757422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c017a940-1438-46eb-8a03-b9aa6c8bd402 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:33.758828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c017a940-1438-46eb-8a03-b9aa6c8bd402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:33.760399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 70/138] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:33 2022] PUT /identity/v3/projects/20b1e025ed514544abb2932a8dace90d/users/87769f448fc54978888c2c57ac8ab26a/roles/b6752aeca4994d6292f5216c4d87e805 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:33.867691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 69/139] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:34.184793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a0b7-bf66-4d8c-b9b6-4776312cc851 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:34.185183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a0b7-bf66-4d8c-b9b6-4776312cc851 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:34.185363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a0b7-bf66-4d8c-b9b6-4776312cc851 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:34.186755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 71/140] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:34.199322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:34.199611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:34.199888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:34.223241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:34.224428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:34.261080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34a0b7-bf66-4d8c-b9b6-4776312cc851 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:34.323720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:34.347365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 70/141] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:34.412587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34a0b7-bf66-4d8c-b9b6-4776312cc851 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:34.451085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:34.451639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:34.451979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:34.452258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:34.453366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:34.455375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:34.456936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 72/142] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:34.462483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:34.500445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:34.571355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c252678-3a8d-4eca-8827-8e978f7b9740 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:34.607834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:34.608446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:34.608727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:34.608986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:34.609659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:34.611195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:34.707404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 71/143] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:34 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:34.909906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:35.040430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abecd-b9df-4fff-93af-a702dd4bc989 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:35.048151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:35.081631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:35.081631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:35.081631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:35.081631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:35.091400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:35.097125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:35.097975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:35.098807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:35.101087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 73/144] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:35.106498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:35.223647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:35.240714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5300a-b646-43a7-9260-3efaf7b7e9b1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:35.291352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:35.291904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:35.292186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:35.292686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:35.293663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:35.293972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:35.295413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:35.305858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 72/145] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:35.311420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:35.452373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5af35-5cbe-4612-b5bc-0cf882ac3112 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:35.503004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cf70c-2bf0-4ec4-a05d-953f5f595d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:35.504016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf70c-2bf0-4ec4-a05d-953f5f595d48 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:35.504479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf70c-2bf0-4ec4-a05d-953f5f595d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:35.504969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf70c-2bf0-4ec4-a05d-953f5f595d48 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:35.506277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cf70c-2bf0-4ec4-a05d-953f5f595d48 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:35.507930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cf70c-2bf0-4ec4-a05d-953f5f595d48 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:35.510039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 74/146] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:35 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:35.515260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:35.643286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b2a71-ab7b-466b-8527-7e56624540df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:35.684461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:35.685357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:35.685830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:35.686281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:35.714175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:35.747760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79486-f95b-493f-b8bc-96aaa2d1c86b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:35.748057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79486-f95b-493f-b8bc-96aaa2d1c86b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:35.748312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79486-f95b-493f-b8bc-96aaa2d1c86b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:35.749645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 75/147] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:55:35.755927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:35.756476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:35.756982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:35.766423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:35.768562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:35.794064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:35.795599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:35.852495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:35.924338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 73/148] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:55:35.927721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 76/149] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:35.936922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:36.008845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:36.056969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a91b2-d509-47f6-8f1d-98145c46ef0c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:36.094000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:36.095210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:36.095492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:36.095767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:36.096923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:36.098394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:36.100075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 74/150] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:35 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:36.106040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:36.229572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ff78-4912-4fef-a593-08796058ef22 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:36.257913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:36.281437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:36.282162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:36.282948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:36.283222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:36.283968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=87769f448fc54978888c2c57ac8ab26a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:36.287427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:36.289159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:36.306180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 77/151] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:36 2022] GET /identity/v3/role_assignments?user.id=87769f448fc54978888c2c57ac8ab26a&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=8a09ae9946bc4e158d491383af34a89f => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:36.429391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13ff3-aacd-4f2c-92f8-f7b91a89f3f9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:36.459184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:36.475009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:36.480919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:36.480919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:36.480919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:36.498808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:36.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:36.505205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:36.505998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 75/152] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:36.511523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:36.546886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:36.631775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ea6d-6a6d-45d3-b893-68aa2c28c779 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:36.676953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:36.677574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:36.677851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:36.678126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:36.678891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:36.679198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:36.680597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:36.690606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 78/153] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:36.696228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:36.827045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a54b06-2046-4deb-ab0c-0909f322d0a1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:36.866864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:36.867772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:36.868194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:36.868620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] PATH_INFO: `/v3/users/4dc813aca93149f799bbc6c108d84521` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:36.869944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=4dc813aca93149f799bbc6c108d84521)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:36.871573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:36.873613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 76/154] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:36 2022] GET /identity/v3/users/4dc813aca93149f799bbc6c108d84521 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:36.879445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:37.044672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:37.054657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698244-0ac2-4fb9-8b09-6a5fcc860f3b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:37.095849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f1382-2a2b-4cee-bd59-c5ebc1cb3b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:37.096422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f1382-2a2b-4cee-bd59-c5ebc1cb3b11 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.096701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f1382-2a2b-4cee-bd59-c5ebc1cb3b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.096977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f1382-2a2b-4cee-bd59-c5ebc1cb3b11 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.098264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f1382-2a2b-4cee-bd59-c5ebc1cb3b11 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:37.099798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f1382-2a2b-4cee-bd59-c5ebc1cb3b11 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:37.101562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 79/155] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:36 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:37.107480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:37.196080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:37.273739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400da5b-685b-47c0-9468-b96e7d519ef7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:37.317511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211710ca-9ff9-476f-95c8-22bfd7243d57 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.317809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211710ca-9ff9-476f-95c8-22bfd7243d57 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.318109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211710ca-9ff9-476f-95c8-22bfd7243d57 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.319566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 80/156] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:55:37.322243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ad016-e8bf-46ba-b331-229c4ef087b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:37.322929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ad016-e8bf-46ba-b331-229c4ef087b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.323362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ad016-e8bf-46ba-b331-229c4ef087b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.323640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ad016-e8bf-46ba-b331-229c4ef087b0 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d/users/4dc813aca93149f799bbc6c108d84521/roles/f448f4dca91e4f1982f116ee3ff052b6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.326144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c5a02-91ae-4f84-80d9-8ea6cf228452 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.326428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c5a02-91ae-4f84-80d9-8ea6cf228452 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.326795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c5a02-91ae-4f84-80d9-8ea6cf228452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.327489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ad016-e8bf-46ba-b331-229c4ef087b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=20b1e025ed514544abb2932a8dace90d, user_id=4dc813aca93149f799bbc6c108d84521, role_id=f448f4dca91e4f1982f116ee3ff052b6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:37.329569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ad016-e8bf-46ba-b331-229c4ef087b0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:37.363717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-071c5a02-91ae-4f84-80d9-8ea6cf228452 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:37.364870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-071c5a02-91ae-4f84-80d9-8ea6cf228452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:37.404639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 77/157] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:37 2022] PUT /identity/v3/projects/20b1e025ed514544abb2932a8dace90d/users/4dc813aca93149f799bbc6c108d84521/roles/f448f4dca91e4f1982f116ee3ff052b6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:37.407442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35bf2-bc98-4ec7-ab11-dcae0e631a1d None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.407759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35bf2-bc98-4ec7-ab11-dcae0e631a1d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.408030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35bf2-bc98-4ec7-ab11-dcae0e631a1d None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.409426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 78/158] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:37.414886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965afef5-bc2f-4970-aa55-4c16e85c6160 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.415165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965afef5-bc2f-4970-aa55-4c16e85c6160 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.415436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965afef5-bc2f-4970-aa55-4c16e85c6160 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.418984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 79/159] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:37.420135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.420992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.420992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.460014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:37.461260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:37.464142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 81/160] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:37.465663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:37.466014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:37.466490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:37.503452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:37.504660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:37.605679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 80/161] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:37.629214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 82/162] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:37.977634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.026598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:38.096062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14795f-f44f-4304-807a-c38182597fe0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.133228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.134020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.134401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.134812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.146185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:38.151176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:38.152123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:38.152888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 81/163] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:38.160046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.168560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.203876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:38.279351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:38.286668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87e08e-ccdf-4f13-b984-e08348771023 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.286939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce38e0c-5172-43fc-93f7-51dd4f38ec64 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.322365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.323478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.323892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.324322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.335961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:38.341186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:38.342269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:38.343202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:38.345729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 83/164] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:38.346881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.348196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.348835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.349112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.349384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.350085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:38.350390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:38.352006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:38.373774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 82/165] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:38.373774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.449167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:38.473253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60ffd2-5b04-4ea5-929e-90076cc288af None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.508299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.509156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.509570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.509976 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.533158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:38.542712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:38.543791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 84/166] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:38.544761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.589848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e2e28-2e76-4048-955b-28d796b6fbd4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.668051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.668679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.668956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.669230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.669907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:38.670210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:38.671761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:38.689012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 83/167] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:38.690027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.698683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a2b11-fe0b-4f38-af7a-791b63a8ae8b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.741202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:38.775365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.776365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.776837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.777300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.778737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:38.780498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:38.783038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 85/168] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:38.785055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:38.796255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:38.916722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6cb6c-e798-4ed1-bf7f-f8a1d2b6c651 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:38.989050 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:38.990104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:38.990590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:38.991038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:38.991884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:38.992362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:38.993938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:39.006268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd6d0-c162-4842-8b5b-74707263f590 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:39.009570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 84/169] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:39.010604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:39.051471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:39.052026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.052325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.052583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] PATH_INFO: `/v3/users/87769f448fc54978888c2c57ac8ab26a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.054071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] RBAC: Authorizing `identity:get_user(user_id=87769f448fc54978888c2c57ac8ab26a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:39.055657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:39.057259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 86/170] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:38 2022] GET /identity/v3/users/87769f448fc54978888c2c57ac8ab26a => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:39.058021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:39.174914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:39.175768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:39.201674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbb461-d336-49bd-8d86-6cdf14e3e53f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:39.238096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c886e-fc7a-43a1-b0a2-7d4497d06e75 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:39.280015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3fb45-3f84-43c1-b2c9-7bd813d1d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:39.280630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3fb45-3f84-43c1-b2c9-7bd813d1d841 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.280912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3fb45-3f84-43c1-b2c9-7bd813d1d841 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.281189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3fb45-3f84-43c1-b2c9-7bd813d1d841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.304357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff6185-29fc-4e53-8e15-ac593dcf8d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:39.305025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff6185-29fc-4e53-8e15-ac593dcf8d03 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.305304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff6185-29fc-4e53-8e15-ac593dcf8d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.305579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff6185-29fc-4e53-8e15-ac593dcf8d03 None admin] PATH_INFO: `/v3/users/bc4a17a3aa754cf9b61c2b367071735b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.306962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff6185-29fc-4e53-8e15-ac593dcf8d03 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4a17a3aa754cf9b61c2b367071735b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:39.308409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff6185-29fc-4e53-8e15-ac593dcf8d03 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:39.310016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 87/171] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:39 2022] GET /identity/v3/users/bc4a17a3aa754cf9b61c2b367071735b => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:39.311191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278c09-702b-4830-85ed-ec3d8b4efbf3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.311472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278c09-702b-4830-85ed-ec3d8b4efbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.311760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278c09-702b-4830-85ed-ec3d8b4efbf3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.313544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 88/172] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:39.314266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278c09-702b-4830-85ed-ec3d8b4efbf3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:39.408324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3fb45-3f84-43c1-b2c9-7bd813d1d841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:39.409801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3fb45-3f84-43c1-b2c9-7bd813d1d841 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:39.476478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278c09-702b-4830-85ed-ec3d8b4efbf3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:39.528681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:39.529309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.529591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.529873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.547007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:39.555843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:39.572552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 85/173] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:39.573945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.578946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.579346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.598114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:39.599490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:39.671609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:39.676682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:39.699203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 86/174] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:39.700074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:39.843421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65554b-b73e-4294-afc7-6e09071b60ce None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:39.882931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:39.920713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5783797-5e80-4b70-9763-d9f37053d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:39.921420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5783797-5e80-4b70-9763-d9f37053d183 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:39.921705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5783797-5e80-4b70-9763-d9f37053d183 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:39.921977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5783797-5e80-4b70-9763-d9f37053d183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:39.928809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 89/175] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:39.929672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.015965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.056277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5783797-5e80-4b70-9763-d9f37053d183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.059508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5783797-5e80-4b70-9763-d9f37053d183 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.100189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e7cc-cd42-4f9b-9d2d-9963802fefe1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.142168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.143324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.143612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.143885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] PATH_INFO: `/v3/projects/897c283a15da4e8eb49018829aa5c06f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.145193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] RBAC: Authorizing `identity:get_project(project_id=897c283a15da4e8eb49018829aa5c06f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.147053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.148324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 90/176] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:39 2022] GET /identity/v3/projects/897c283a15da4e8eb49018829aa5c06f => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:40.151391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.180416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 87/177] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:40.186104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb60a-9496-44a6-a2a7-96dc52e76dcb None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.186386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb60a-9496-44a6-a2a7-96dc52e76dcb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.186864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb60a-9496-44a6-a2a7-96dc52e76dcb None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.191985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 88/178] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:40.193084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008330-3f02-400f-a6cb-a799a89d0894 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.193361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008330-3f02-400f-a6cb-a799a89d0894 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.193634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008330-3f02-400f-a6cb-a799a89d0894 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.194991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 89/179] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:40.199304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008330-3f02-400f-a6cb-a799a89d0894 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.225320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.266102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a38a8-fb9c-4b18-9fbc-681508f905b2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.277620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.301890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.302519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.302822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.303089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.304267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.305802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.307827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 91/180] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:40.308932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.323229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008330-3f02-400f-a6cb-a799a89d0894 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.359571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.360819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.361092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.361361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.372075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:40.376930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:40.377807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 90/181] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:40.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.428208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82546dd7-9f2f-44f2-a6a8-7c9f2612ccb0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.464673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.466042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.466328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.466629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.467365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=897c283a15da4e8eb49018829aa5c06f, user.id=87769f448fc54978888c2c57ac8ab26a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:40.467676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.469111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6b531-802e-4f27-aecc-1d751e21a6b8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.472604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.477159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 92/182] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/role_assignments?user.id=87769f448fc54978888c2c57ac8ab26a&scope.project.id=897c283a15da4e8eb49018829aa5c06f&role.id=8a09ae9946bc4e158d491383af34a89f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:40.480137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.480997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.481480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.497766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40236fe3-dd56-4667-96eb-346baf5b5730 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.509404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:40.511983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:40.541274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b963556-b0a8-4abc-97dd-c4fdb89a3eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.541933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b963556-b0a8-4abc-97dd-c4fdb89a3eef None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.542210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b963556-b0a8-4abc-97dd-c4fdb89a3eef None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.542481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b963556-b0a8-4abc-97dd-c4fdb89a3eef None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.543744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b963556-b0a8-4abc-97dd-c4fdb89a3eef None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.545228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b963556-b0a8-4abc-97dd-c4fdb89a3eef None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.551066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 91/183] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:40.551066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.551066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.551066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.558881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.569919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:40.571185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:40.596178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 93/184] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:40.597486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.665065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 92/185] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:40.667970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.725703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcec4ae-fe00-49c7-8eac-b03493918599 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.764468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.769555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.770438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.771124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.771977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.773084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] RBAC: Adding query filter params (role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=87769f448fc54978888c2c57ac8ab26a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:40.773755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.775421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2d8b5-de64-490b-845b-c918190f683a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.782253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 94/186] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/role_assignments?user.id=87769f448fc54978888c2c57ac8ab26a&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:40.788700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3206-f63e-4979-866a-3e49a29323e5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.788700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3206-f63e-4979-866a-3e49a29323e5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.788700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3206-f63e-4979-866a-3e49a29323e5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.788999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 95/187] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:40.790205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3206-f63e-4979-866a-3e49a29323e5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:40.814337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7382a24-46da-4b8e-b248-d43da3cf394e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:40.876183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:40.884890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:40.891692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.891692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.891692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.891692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:40.891692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:40.891692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f38228-75f7-4b76-8809-be6d7ad2c537 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:40.898854 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 93/188] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:40.900416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:40.900797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:40.901083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:40.925220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:40.926354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:40.979292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3206-f63e-4979-866a-3e49a29323e5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.017353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 94/189] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:41.018177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.023309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.024381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.024381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.024577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.026064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=bc4a17a3aa754cf9b61c2b367071735b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:41.026064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.027535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.038104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 96/190] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:40 2022] GET /identity/v3/role_assignments?user.id=bc4a17a3aa754cf9b61c2b367071735b&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:41.039012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.110960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.155969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be36c8-bdc2-47ef-a99b-b17c8ee4a117 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.194348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.198808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29f32-b226-4421-80a7-47308aca8896 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.206057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.207068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.207537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.207820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.221839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:41.228898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:41.229847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:41.230665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 95/191] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:41.234415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.246358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.247389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.247874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.248859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] PATH_INFO: `/v3/users/8f1d35673169426c920dede8c8c8ea1f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.269104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1d35673169426c920dede8c8c8ea1f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.271576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.273734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 97/192] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/users/8f1d35673169426c920dede8c8c8ea1f => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:41.275378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.358056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bbac3-35b1-4d32-abf9-7dd7bf5bc6a0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.394499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.400082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68797-3596-4507-a963-dbb0dfffcf17 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.403137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.406226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.406226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.406226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.406226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.407443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.409030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 96/193] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:41.409798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.434965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.435567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.435843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.436115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.436774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:41.437071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.438467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.446222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 98/194] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:41.446968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.468925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.509009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156af82-3d31-4968-b5f8-1706b51cdce9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.544402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.545000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.545275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.545552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.548262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c97f3-0778-41ea-a553-7c24b97da37d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.581013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.581623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.581896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.582164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.592001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:41.597198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:41.598067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:41.599189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 99/195] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:41.599995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.609026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.615953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.617471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.678103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.699487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fc468-54e1-4e63-a6d6-4f0051422776 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.722158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 97/196] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:41.723129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.735224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.735828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.736105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.736379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.737043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.738479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.813451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:41.820232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da0f9e-97a2-4b77-a686-082f02446614 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:41.852771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:41.853494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:41.853769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:41.854041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] PATH_INFO: `/v3/users/8f1d35673169426c920dede8c8c8ea1f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:41.855153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] RBAC: Authorizing `identity:get_user(user_id=8f1d35673169426c920dede8c8c8ea1f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:41.856602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:41.858171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 98/197] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/users/8f1d35673169426c920dede8c8c8ea1f => generated 283 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:41.858999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.878576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 100/198] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:41 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:41.878576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:41.968324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978b58-79b4-4284-b0ad-dcd241ce9ade None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.002482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a163d8-f405-4d08-bbd6-2d64369d1df0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.007343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.012075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.012075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.012075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.012075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:42.012075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.012075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.025323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 99/199] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:42.025323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:42.048114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.048751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.049034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.049346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.050671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:42.051445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.052892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.054861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 101/200] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:41 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:42.055638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:42.059174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:42.137368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41203c5-4baf-4a26-ab5b-96dc82224509 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.177397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f7f8-d079-4fec-9f7f-d2e7983e7918 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.182642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.183265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.183540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.183807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.220348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.220997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.221277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.221575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] PATH_INFO: `/v3/users/4dc813aca93149f799bbc6c108d84521` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.223000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] RBAC: Authorizing `identity:get_user(user_id=4dc813aca93149f799bbc6c108d84521)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.224468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.227064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 102/201] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/users/4dc813aca93149f799bbc6c108d84521 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:42.227064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:42.273520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.275095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.316930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:42.337657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:42.375057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc0390-d4ae-4322-bdc5-2f8b545cb5a3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.405975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 100/202] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:42.408780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:42.421020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.422024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.423161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.423983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.425231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=8f1d35673169426c920dede8c8c8ea1f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:42.426026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.427710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.434342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 103/203] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/role_assignments?user.id=8f1d35673169426c920dede8c8c8ea1f&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:42.435414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:42.552577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5443cf-97a8-4da9-ba27-c83cc2973deb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.561882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cd2a1-5eb8-40a9-aa7d-7b0508df88ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.593890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:42.618512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.619160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.619486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.619757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.626729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.631227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.631557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.631766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.634052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.635543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.637313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 104/204] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:42.645466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:42.650357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:42.748462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.749943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.836219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5944cd0-4107-44f7-8632-610bbfa05261 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:42.899863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:42.900472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:42.900747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:42.901012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:42.901741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] RBAC: Adding query filter params (role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=8f1d35673169426c920dede8c8c8ea1f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:42.902044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:42.903776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:42.909527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 105/205] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/role_assignments?user.id=8f1d35673169426c920dede8c8c8ea1f&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:42.939376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 101/206] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:42.945029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:43.060547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:43.111759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff8293-2749-4ca8-bb6f-d597e442ee53 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:43.153391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d71686-5b19-4d15-8ddc-4c43df9ba53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:43.154018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71686-5b19-4d15-8ddc-4c43df9ba53b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:43.154292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71686-5b19-4d15-8ddc-4c43df9ba53b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:43.154575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71686-5b19-4d15-8ddc-4c43df9ba53b None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:43.155746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71686-5b19-4d15-8ddc-4c43df9ba53b None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:43.157243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71686-5b19-4d15-8ddc-4c43df9ba53b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:43.159247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 106/207] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:42 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:43.165428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:43.216797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:43.333027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6f196-c1f2-40af-bd6b-a4583dd92087 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:43.374832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:43.375943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:43.376285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:43.376565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:43.377360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] RBAC: Adding query filter params (role.id=f448f4dca91e4f1982f116ee3ff052b6, scope.project.id=20b1e025ed514544abb2932a8dace90d, user.id=4dc813aca93149f799bbc6c108d84521) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:43.377671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:43.379202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2f0ef-f667-4fa4-af35-68bd7c7d6350 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:43.391539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 102/208] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:43 2022] GET /identity/v3/role_assignments?user.id=4dc813aca93149f799bbc6c108d84521&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=f448f4dca91e4f1982f116ee3ff052b6 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:44.512198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c080b0-c3b2-4d6d-9887-c953bec4da75 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:44.512198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c080b0-c3b2-4d6d-9887-c953bec4da75 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:44.512198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c080b0-c3b2-4d6d-9887-c953bec4da75 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:44.512198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 107/209] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:44.518266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199f0bb-759c-420f-8a28-bf8ce738c9e0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:44.519078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199f0bb-759c-420f-8a28-bf8ce738c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:44.519554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199f0bb-759c-420f-8a28-bf8ce738c9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:44.538430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1199f0bb-759c-420f-8a28-bf8ce738c9e0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:44.539900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1199f0bb-759c-420f-8a28-bf8ce738c9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:44.584671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69798cc4-3659-4588-a3f9-55b8ada8e55b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:44.585287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69798cc4-3659-4588-a3f9-55b8ada8e55b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:44.585754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69798cc4-3659-4588-a3f9-55b8ada8e55b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:44.587696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 108/210] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:44.596087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd357-789d-428a-a085-8ea0870ae7d6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:44.596646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd357-789d-428a-a085-8ea0870ae7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:44.597129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd357-789d-428a-a085-8ea0870ae7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:44.615939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1e8dd357-789d-428a-a085-8ea0870ae7d6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:44.617217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 103/211] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:44.618467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1e8dd357-789d-428a-a085-8ea0870ae7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:44.700033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 109/212] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:44.780872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacdd22-8c99-424c-92ae-fd4a3e7f5e46 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:44.781171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacdd22-8c99-424c-92ae-fd4a3e7f5e46 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:44.781866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacdd22-8c99-424c-92ae-fd4a3e7f5e46 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:44.783522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 104/213] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:44.789032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea09024-25f5-403a-add7-e921219c8463 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:44.789318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea09024-25f5-403a-add7-e921219c8463 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:44.789592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea09024-25f5-403a-add7-e921219c8463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:44.808601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eea09024-25f5-403a-add7-e921219c8463 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:44.809778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eea09024-25f5-403a-add7-e921219c8463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:44.875187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:44.883566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 110/214] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:44.907555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacdd22-8c99-424c-92ae-fd4a3e7f5e46 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:44.929288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:44.973262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea09024-25f5-403a-add7-e921219c8463 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.030097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacdd22-8c99-424c-92ae-fd4a3e7f5e46 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.073333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.073890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.074178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.074454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.076078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:45.077851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:45.080601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 105/215] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:45.085903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.091650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea09024-25f5-403a-add7-e921219c8463 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.129932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.131028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.131520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.131990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.144198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:45.149990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:45.151025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:45.151845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:45.159602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 111/216] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:55:45.160186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.179245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:45.194675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975ae1b-b4bc-4433-ba7d-56b7c5b96df4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.226517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:45.235559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.236178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.236458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.236732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.237580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:45.238865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:45.272969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700c655-5734-44e2-9d5b-a02634b674d2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.307106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47193eb4-a346-4a8a-9e15-c2b505a900cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.307726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47193eb4-a346-4a8a-9e15-c2b505a900cb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.308002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47193eb4-a346-4a8a-9e15-c2b505a900cb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.308271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47193eb4-a346-4a8a-9e15-c2b505a900cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.318718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47193eb4-a346-4a8a-9e15-c2b505a900cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:45.322350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 106/217] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:45 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:45.323301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-47193eb4-a346-4a8a-9e15-c2b505a900cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:45.331525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:45.332419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 112/218] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:45.333239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe549b-1dc0-4f9b-91e5-59bf78d7eebe None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.334498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe549b-1dc0-4f9b-91e5-59bf78d7eebe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.334498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe549b-1dc0-4f9b-91e5-59bf78d7eebe None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.334498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 113/219] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:45.334498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe549b-1dc0-4f9b-91e5-59bf78d7eebe None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.446105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a53da-234d-471e-93c9-29421ccc1f3e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.451986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe549b-1dc0-4f9b-91e5-59bf78d7eebe None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.472955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:45.482049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.485840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.485840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.485840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.485840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:45.485840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:45.485840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf150-55d1-4cbe-a6ab-b6676b8508e2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:45.493730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.494282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 107/220] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:45.494387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.494625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.494896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.495539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.495649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:45.495836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.495929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:45.496079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.497271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:45.507719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:45.509307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 114/221] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:45.510150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.520422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:45.521600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:45.603252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 108/222] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:45.604046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.639474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21746d62-57d9-4da9-84c4-cc89da0ad3fe None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.694185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.695225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.695225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.695470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] PATH_INFO: `/v3/users/9ee3e17b3ac64517937fc782b2e0b10c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.710867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] RBAC: Authorizing `identity:get_user(user_id=9ee3e17b3ac64517937fc782b2e0b10c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:45.712321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:45.713999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 115/223] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/users/9ee3e17b3ac64517937fc782b2e0b10c => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:45.719714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.743637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7337b-b867-494d-a564-c3347f15c497 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.762257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:45.793728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.794380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.794684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.795030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] PATH_INFO: `/v3/users/8f1d35673169426c920dede8c8c8ea1f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.797413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1d35673169426c920dede8c8c8ea1f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:45.798919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:45.800685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 109/224] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/users/8f1d35673169426c920dede8c8c8ea1f => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:45.806290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:45.833697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:45.863272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9448c-28b9-4c64-b5b6-2ee89b4307e9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:45.901050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:45.902829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:45.903106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:45.903376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:45.974494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33080c-85eb-4c8d-a64f-6e6878781c37 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.000137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.002011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.016652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.017287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.017566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.017836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.019128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.021445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.024509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 110/225] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:46.025530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.047354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:46.106929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:46.139533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 116/226] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:46.140382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.142773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422cb8e-76c8-4158-ba78-f8cfe055a463 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.179812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.180439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.180711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.180979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.192227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:46.197229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:46.198093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:46.198912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 111/227] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:55:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:46.199579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.251434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295bebe-9d7e-414d-82ac-294c6fd9a4aa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.289592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.290354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.290676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.290956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909/users/8f1d35673169426c920dede8c8c8ea1f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.293008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=19d2ad19ff1b486589350b08b8845909, user_id=8f1d35673169426c920dede8c8c8ea1f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.294622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.298271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:46.314569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c3c77-5600-450d-a4ee-9474eb73cbec None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.345994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:46.353291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.353980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.354255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.354523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.355672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.357261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.363681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 112/228] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:46 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:46.363681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.413778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 117/229] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:46 2022] PUT /identity/v3/projects/19d2ad19ff1b486589350b08b8845909/users/8f1d35673169426c920dede8c8c8ea1f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:46.415568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.507631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a08367-65f3-4d4f-a8f7-49803c035bfa None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.535330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660f689-aa1b-4030-a792-331d9adbb1c3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.556644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.557351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.557864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.558571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:46.558865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.560283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.575265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 113/230] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:55:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:46.580405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.587381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.589323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.590921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.591974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.592947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=9ee3e17b3ac64517937fc782b2e0b10c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:46.597186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.599063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.610130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 118/231] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:46 2022] GET /identity/v3/role_assignments?user.id=9ee3e17b3ac64517937fc782b2e0b10c&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=8a09ae9946bc4e158d491383af34a89f => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:46.679530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:46.725753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b294b-0e64-4f4f-a59e-809d02597b6f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.767611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.768266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.768545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.768821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] PATH_INFO: `/v3/users/8f1d35673169426c920dede8c8c8ea1f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.770018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1d35673169426c920dede8c8c8ea1f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.771675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.773565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 114/232] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:46 2022] GET /identity/v3/users/8f1d35673169426c920dede8c8c8ea1f => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:46.779327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:46.870226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:46.919387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3860-1f2f-4138-ab6c-75578da11aa6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:46.969365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed4372-7db4-455d-9f52-027868198aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:46.969921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed4372-7db4-455d-9f52-027868198aca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:46.970194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed4372-7db4-455d-9f52-027868198aca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:46.970466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed4372-7db4-455d-9f52-027868198aca None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:46.972977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed4372-7db4-455d-9f52-027868198aca None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:46.974427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed4372-7db4-455d-9f52-027868198aca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:46.976396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 119/233] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:46 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:46.981881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:47.106326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c4d7c-62a7-4e6d-a839-d73f41b92a18 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:47.133957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:47.142640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c029be9-b960-46a9-9960-41739cf7d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:47.145895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c029be9-b960-46a9-9960-41739cf7d0de None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:47.145895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c029be9-b960-46a9-9960-41739cf7d0de None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:47.145895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c029be9-b960-46a9-9960-41739cf7d0de None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909/users/8f1d35673169426c920dede8c8c8ea1f/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:47.146102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c029be9-b960-46a9-9960-41739cf7d0de None admin] RBAC: Authorizing `identity:create_grant(project_id=19d2ad19ff1b486589350b08b8845909, user_id=8f1d35673169426c920dede8c8c8ea1f, role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:47.147525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c029be9-b960-46a9-9960-41739cf7d0de None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:47.271656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 115/234] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:46 2022] PUT /identity/v3/projects/19d2ad19ff1b486589350b08b8845909/users/8f1d35673169426c920dede8c8c8ea1f/roles/b6752aeca4994d6292f5216c4d87e805 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 18:55:47.820254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b684-41a0-4bab-80c3-a6268655b9c9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:47.820807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b684-41a0-4bab-80c3-a6268655b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:47.820807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b684-41a0-4bab-80c3-a6268655b9c9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:47.822441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 120/235] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:47.835978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea2b5e-8948-4068-ae23-3a854cf809b2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:47.836412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea2b5e-8948-4068-ae23-3a854cf809b2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:47.836594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea2b5e-8948-4068-ae23-3a854cf809b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:47.858925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-98ea2b5e-8948-4068-ae23-3a854cf809b2 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:47.860257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-98ea2b5e-8948-4068-ae23-3a854cf809b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:47.939348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 116/236] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:48.111867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95613cbe-0041-4b55-b4bc-fb6f8629e20d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:48.111867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95613cbe-0041-4b55-b4bc-fb6f8629e20d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:48.111867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95613cbe-0041-4b55-b4bc-fb6f8629e20d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:48.111867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 121/237] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:48.117720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:48.118001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:48.118275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:48.138661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:48.139869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:48.215383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 117/238] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:48.402132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95613cbe-0041-4b55-b4bc-fb6f8629e20d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:48.539795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95613cbe-0041-4b55-b4bc-fb6f8629e20d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:48.588852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:48.601940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:48.615319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:48.615319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:48.615319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:48.627564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:48.632464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:48.633343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:48.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 122/239] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:48.639456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:48.686610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:48.722427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:48.811362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038fbb3-6c00-4841-83d8-bd1d1be0d64e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:48.862781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd2061-770b-4c13-a3ba-04f019db76fb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:48.899607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:48.900258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:48.900493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:48.900760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:48.901568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:48.902184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:48.902449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:48.902736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:48.903344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:48.903673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:48.905106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:48.907334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:48.908785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:48.910411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 123/240] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:48.916345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:48.923249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 118/241] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:48.931545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:48.992754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:49.063044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535c25b-4f22-44ab-9a73-00d08cddd0d4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:49.110636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbba72e-170d-4235-aeb8-27c478024de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:49.111268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba72e-170d-4235-aeb8-27c478024de7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.111542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba72e-170d-4235-aeb8-27c478024de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.111816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba72e-170d-4235-aeb8-27c478024de7 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.112499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbba72e-170d-4235-aeb8-27c478024de7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:49.114000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbba72e-170d-4235-aeb8-27c478024de7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:49.143553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df8f3-675e-4616-b1f9-0c141226cd5d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:49.154840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:49.179263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:49.179988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.180271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.180543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] PATH_INFO: `/v3/users/48044362b1ab4a76acbdd5d1fc56b9ae` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.194992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] RBAC: Authorizing `identity:get_user(user_id=48044362b1ab4a76acbdd5d1fc56b9ae)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:49.196573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:49.198616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 119/242] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:48 2022] GET /identity/v3/users/48044362b1ab4a76acbdd5d1fc56b9ae => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:49.209660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:49.221460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 124/243] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:48 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:49.275372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a61da-f8ee-4e5d-9577-1ecec11a6139 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.275958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a61da-f8ee-4e5d-9577-1ecec11a6139 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.276402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a61da-f8ee-4e5d-9577-1ecec11a6139 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.278114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 125/244] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:55:49.286318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e5bfd-2fda-49bc-9e01-3266b04bb001 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.286621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e5bfd-2fda-49bc-9e01-3266b04bb001 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.286947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e5bfd-2fda-49bc-9e01-3266b04bb001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.323404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0b1e5bfd-2fda-49bc-9e01-3266b04bb001 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:49.325131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0b1e5bfd-2fda-49bc-9e01-3266b04bb001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:49.385224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570365-f6d1-46bc-9b5a-d687a9c3c76f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:49.434284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:49.437130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.437130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.437130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.453948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 126/245] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:49.453948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c50738-dffb-4f33-ba37-6c1934a1ce89 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.453948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c50738-dffb-4f33-ba37-6c1934a1ce89 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.453948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c50738-dffb-4f33-ba37-6c1934a1ce89 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.454487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 127/246] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:49.464200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.464479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.464749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.481033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:49.487101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:49.488286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:49.518505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:49.520319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:49.530468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:49.573601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 128/247] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:49.637763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 120/248] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:49.643402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:49.755926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd4d89-9bbe-4009-a65b-be3ae3fc11ef None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:49.772307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:49.796715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:49.797692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.797964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.798228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.799646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:49.801764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:49.804182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 129/249] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:49 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:49.809915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:49.851089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:49.860335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:49.895388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c90cd-47d7-45bc-8f4c-76ca3006d7b3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:49.940222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a957d5b-0cfa-4574-a604-ca9e3e6521bb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:49.946297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:49.946990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.947270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.947542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.963554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:49.969269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:49.970144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:55:49.970967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:49.972889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 121/250] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:49.978332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:49.989898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:49.990508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:49.990820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:49.991084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:49.991809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] RBAC: Adding query filter params (role.id=f448f4dca91e4f1982f116ee3ff052b6, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=48044362b1ab4a76acbdd5d1fc56b9ae) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:49.992107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:49.993733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:50.001404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 130/251] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:49 2022] GET /identity/v3/role_assignments?user.id=48044362b1ab4a76acbdd5d1fc56b9ae&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=f448f4dca91e4f1982f116ee3ff052b6 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:50.002324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:50.148293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc62420-5dac-42ee-bd23-88b1011bb829 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:50.152827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:50.171120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2744dd-fb66-49b8-af43-704b3b4c6ef2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:50.219139 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:50.219627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.219901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.220217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.220894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:50.221191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:50.222814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8a3b-692b-49d9-9adb-60eb618e3069 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:50.233963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:50.241741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 131/252] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:50.242083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54752a5d-cc56-421b-ae68-92df07c94dc1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.242367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54752a5d-cc56-421b-ae68-92df07c94dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.243021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54752a5d-cc56-421b-ae68-92df07c94dc1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.245548 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c62afd-51a6-42b7-be22-708918979e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:50.246183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62afd-51a6-42b7-be22-708918979e38 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.246456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62afd-51a6-42b7-be22-708918979e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.250898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62afd-51a6-42b7-be22-708918979e38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.251853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 132/253] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:50.252704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54752a5d-cc56-421b-ae68-92df07c94dc1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:50.279192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c62afd-51a6-42b7-be22-708918979e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-12c62afd-51a6-42b7-be22-708918979e38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:50.284161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:50.285023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:50.285819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 122/254] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:50.287040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.287320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.287845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.316364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:50.318228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:50.432909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 123/255] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:50.433500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:50.444750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54752a5d-cc56-421b-ae68-92df07c94dc1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:50.528229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:50.529255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.529728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.530184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] PATH_INFO: `/v3/users/8f1d35673169426c920dede8c8c8ea1f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.534388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1d35673169426c920dede8c8c8ea1f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:50.536161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:50.538260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 133/256] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:50 2022] GET /identity/v3/users/8f1d35673169426c920dede8c8c8ea1f => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:50.547066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:50.548708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440dfc1-68b6-431d-8599-ca69038a8394 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:50.567471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:50.586808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:50.587695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.588171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.588511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.589818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:50.590189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:50.591878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:50.592037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:50.606414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 124/257] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:50.611936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:50.668714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b66e5-a291-4d09-8493-4191e5a09a42 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:50.706639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:50.707167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.707444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.707717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.745389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9c39c-761c-4b80-b852-3a1ddc843252 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:50.786835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:50.788357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:50.788813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:50.789239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] PATH_INFO: `/v3/users/9ee3e17b3ac64517937fc782b2e0b10c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:50.790667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=9ee3e17b3ac64517937fc782b2e0b10c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:50.792670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:50.794740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 125/258] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:50 2022] GET /identity/v3/users/9ee3e17b3ac64517937fc782b2e0b10c => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:50.801645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:50.812665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:50.814243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:50.862434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:50.901487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:50.928335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07264-5692-4416-ae12-f69c8d0d0cb3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.006037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.006697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.006887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.007156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.008535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:51.010003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:51.011678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 126/259] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:50 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:51.012449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.033015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 134/260] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:51.033852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.141788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6996c-f87a-4afa-bf52-f32ec2f3cf39 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.177556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.178250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.178533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.178841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909/users/9ee3e17b3ac64517937fc782b2e0b10c/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.181219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=19d2ad19ff1b486589350b08b8845909, user_id=9ee3e17b3ac64517937fc782b2e0b10c, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:51.182736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:51.187575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:51.246529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e164-bcdd-4b19-9f43-cc8be23e6083 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.247254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 135/261] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:51 2022] PUT /identity/v3/projects/19d2ad19ff1b486589350b08b8845909/users/9ee3e17b3ac64517937fc782b2e0b10c/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:51.248439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.281400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:51.336272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.336926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.337223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.337871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.364473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91bfb3-8fb1-4bf8-a723-c62cc37faba8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.365783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:51.370897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:51.371832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:51.372630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 127/262] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:55:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:51.378942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.405804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.407521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.408014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.408501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.409968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:51.411660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:51.414089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 136/263] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:51 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:51.420578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.506908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:51.530241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4142199-93a1-4bee-b7bc-059642c46c1a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.573318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.574723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.575165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.575613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.576512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=8f1d35673169426c920dede8c8c8ea1f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:51.576977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:51.581819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:51.589334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 137/264] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:51 2022] GET /identity/v3/role_assignments?user.id=8f1d35673169426c920dede8c8c8ea1f&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:55:51.666701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eb3b9-1c50-454a-bc50-49611f862dea None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.710052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6678195-6293-4f36-a853-de956956717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.710721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6678195-6293-4f36-a853-de956956717f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.711000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6678195-6293-4f36-a853-de956956717f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.711273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6678195-6293-4f36-a853-de956956717f None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.711961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6678195-6293-4f36-a853-de956956717f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:51.712329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6678195-6293-4f36-a853-de956956717f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:51.713818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6678195-6293-4f36-a853-de956956717f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:51.725512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 128/265] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:55:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:51.731653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.879425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612009a8-a253-49bd-b63e-08e2275f2c77 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:51.913911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:51.924166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:51.924805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:51.925077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:51.932877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] PATH_INFO: `/v3/users/8f1d35673169426c920dede8c8c8ea1f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:51.932877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] RBAC: Authorizing `identity:get_user(user_id=8f1d35673169426c920dede8c8c8ea1f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:51.932877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:51.932877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 138/266] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:51 2022] GET /identity/v3/users/8f1d35673169426c920dede8c8c8ea1f => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:51.935452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6678195-6293-4f36-a853-de956956717f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:51.977924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:52.043437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6678195-6293-4f36-a853-de956956717f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:52.087115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:52.087956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.088370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.088777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.171278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:52.173065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:52.225356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:52.294458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 129/267] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:55:52.300339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:52.414018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fd4e-1e91-44c6-855a-fbfd6921115f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:52.454833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f23cf-928e-4ec8-a5e9-294f5393371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:52.455695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f23cf-928e-4ec8-a5e9-294f5393371e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.456110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f23cf-928e-4ec8-a5e9-294f5393371e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.456532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f23cf-928e-4ec8-a5e9-294f5393371e None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.457784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f23cf-928e-4ec8-a5e9-294f5393371e None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:52.459444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f23cf-928e-4ec8-a5e9-294f5393371e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:52.461502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 139/268] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:52 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:52.467200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:52.608499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca8112-1d41-4347-93f1-30a6b3b6b4ff None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:52.655299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:52.655954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.656233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.656508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.657251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] RBAC: Adding query filter params (role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=8f1d35673169426c920dede8c8c8ea1f) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:52.657558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:52.659040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb91e6-784f-42f8-82d9-36192d459bf3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:52.665701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 130/269] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:52 2022] GET /identity/v3/role_assignments?user.id=8f1d35673169426c920dede8c8c8ea1f&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:52.821561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8df43-691a-46da-a168-fb26c72979ca None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.822249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8df43-691a-46da-a168-fb26c72979ca None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.822769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8df43-691a-46da-a168-fb26c72979ca None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.824643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 140/270] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:52.830519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040df22-c6c2-4c98-ad95-ba7df684cd4b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.831151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040df22-c6c2-4c98-ad95-ba7df684cd4b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.831672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040df22-c6c2-4c98-ad95-ba7df684cd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.840404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:52.848589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cd08-96e4-4dd2-b1c7-f60557f7a1b7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.848907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cd08-96e4-4dd2-b1c7-f60557f7a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.849126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cd08-96e4-4dd2-b1c7-f60557f7a1b7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.850875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 141/271] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:52.857685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687812-5ade-4f70-85e5-38b91d242d85 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:52.857965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687812-5ade-4f70-85e5-38b91d242d85 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:52.859743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687812-5ade-4f70-85e5-38b91d242d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:52.863771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2040df22-c6c2-4c98-ad95-ba7df684cd4b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:52.865100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2040df22-c6c2-4c98-ad95-ba7df684cd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:52.878624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ae687812-5ade-4f70-85e5-38b91d242d85 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:52.880025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae687812-5ade-4f70-85e5-38b91d242d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:52.950995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:52.955025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 131/272] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:52.962981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 142/273] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:53.020402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad64107-a4d7-4389-889f-645e004b8986 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.020718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad64107-a4d7-4389-889f-645e004b8986 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.020961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad64107-a4d7-4389-889f-645e004b8986 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.022587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 132/274] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:53.028218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.028506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.028779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.052264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:53.053441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:53.132951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 143/275] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:53.253409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad64107-a4d7-4389-889f-645e004b8986 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.276693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.381642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad64107-a4d7-4389-889f-645e004b8986 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.391439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e53a51-255c-4e3b-a695-7408b6000f7c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.415474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:53.422320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:53.427347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.428888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.429347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.429936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.436449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.437685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.437952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.438223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.443932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:53.448650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:53.449647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:53.450799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:53.451791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:53.453703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 133/276] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:53.453873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:53.457480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:53.458429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 144/277] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:53.459474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.460196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.563411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041832b-a29c-434d-9e8a-33ae20aa2639 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.574408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f6b27-5504-4a49-9320-3bb336f220fd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.596777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.597406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.597683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.597954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.599094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:53.600568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:53.606401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 134/278] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:53.606401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.616287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.616894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.617172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.617440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.618110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:53.618409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:53.619808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:53.628716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 145/279] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:53.629473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.676424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:53.682239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:53.720377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e7950-4840-4bc7-8f9e-3cedbaed4a44 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.736989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee845f0-d51f-4d90-91f8-531ad2feb308 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.756869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.757482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.757752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.758019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.758708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:53.759008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:53.760373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:53.771307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 135/280] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:53.774666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.777801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.778422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.778783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.779059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.780096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:53.781195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:53.876493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c01887-84a6-4c76-9f76-17431baeefe4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:53.898689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:53.905266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 146/281] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:55:53 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:53.906380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:53.922251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:53.922978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:53.923262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:53.923536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] PATH_INFO: `/v3/users/ec587c55b10f4d74bb39cdbf31eb89dd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:53.945214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] RBAC: Authorizing `identity:get_user(user_id=ec587c55b10f4d74bb39cdbf31eb89dd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:53.945214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:53.949812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 136/282] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/users/ec587c55b10f4d74bb39cdbf31eb89dd => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:53.950836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:54.052808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:54.056930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099eebc-3538-432d-ae1b-071379d68e3b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:54.074006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3d0ac-d3c6-4f2d-8c11-4528e5820080 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:54.111383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:54.112792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.113826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.114332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] PATH_INFO: `/v3/users/48044362b1ab4a76acbdd5d1fc56b9ae` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.115842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] RBAC: Authorizing `identity:get_user(user_id=48044362b1ab4a76acbdd5d1fc56b9ae)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:54.117351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:54.118786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:54.119088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 147/283] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/users/48044362b1ab4a76acbdd5d1fc56b9ae => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:54.119757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.120031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.120307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.121062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, user.id=ec587c55b10f4d74bb39cdbf31eb89dd, scope.system=all) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:54.121370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:54.127125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517d81a-a404-4606-9da6-aba61d9104f2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:54.130253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:54.137648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 137/284] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Sep 7 18:55:53 2022] GET /identity/v3/role_assignments?user.id=ec587c55b10f4d74bb39cdbf31eb89dd&scope.system=all&role.id=8a09ae9946bc4e158d491383af34a89f => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:54.241161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da541cd-f027-4efa-a4be-46112d225cf4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.241754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da541cd-f027-4efa-a4be-46112d225cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.242256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da541cd-f027-4efa-a4be-46112d225cf4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.244059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 138/285] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:54.245590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ca30e-020f-448a-bd02-1eb007bd38d3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:54.251314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.251601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.251852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.281833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:54.283192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:54.301551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:54.302399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.302747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.303506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.305008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:54.306499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:54.309157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 148/286] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:54 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:54.315160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:54.340557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:54.361431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:54.387419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 139/287] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:54.441394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4803-8f7a-4a8c-b266-0f13c5d09074 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:54.477761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:54.478823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.479311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.479768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909/users/48044362b1ab4a76acbdd5d1fc56b9ae/roles/f448f4dca91e4f1982f116ee3ff052b6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.481861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=19d2ad19ff1b486589350b08b8845909, user_id=48044362b1ab4a76acbdd5d1fc56b9ae, role_id=f448f4dca91e4f1982f116ee3ff052b6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:54.483506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:54.552030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 149/288] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:55:54 2022] PUT /identity/v3/projects/19d2ad19ff1b486589350b08b8845909/users/48044362b1ab4a76acbdd5d1fc56b9ae/roles/f448f4dca91e4f1982f116ee3ff052b6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:54.662215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:54.800572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1c197-bb2c-46a7-9fab-f0a912438c93 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:54.839235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:54.839871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:54.840149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:54.840415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:54.852180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:54.857067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:54.857931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:54.858889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 140/289] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:54.865157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:54.981443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:54.988484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0189a2-8ce9-4679-b055-4e47571a33b9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:55.030049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:55.031712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:55.032206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:55.032667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:55.033528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:55.034018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:55.035666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:55.045611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 150/290] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:55.051633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:55.119998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:55.219498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1005d-2cbb-4a08-8965-4722a8704c7e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:55.266320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:55.266976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:55.267252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:55.267523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] PATH_INFO: `/v3/users/9ee3e17b3ac64517937fc782b2e0b10c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:55.271221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] RBAC: Authorizing `identity:get_user(user_id=9ee3e17b3ac64517937fc782b2e0b10c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:55.272689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:55.274279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 141/291] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:55 2022] GET /identity/v3/users/9ee3e17b3ac64517937fc782b2e0b10c => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:55.280191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:55.463976 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96562b-c229-4b0c-b628-9797cbc59477 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:55.507184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:55.507795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:55.508074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:55.508348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:55.542045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:55.596164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:55.597622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:55.720216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 151/292] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:55.725496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:55.861681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15a5c4-9db6-42ee-955d-a506475ec8db None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:55.863242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:55.904366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb61ed-1c2e-4886-b6b2-d34a2c28015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:55.904991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb61ed-1c2e-4886-b6b2-d34a2c28015d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:55.905263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb61ed-1c2e-4886-b6b2-d34a2c28015d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:55.905549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb61ed-1c2e-4886-b6b2-d34a2c28015d None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:55.906688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb61ed-1c2e-4886-b6b2-d34a2c28015d None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:55.908124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb61ed-1c2e-4886-b6b2-d34a2c28015d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:55.910046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 142/293] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:55 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:55.915845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:56.018728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:56.069557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae9ca1-487d-4178-847e-037a13d0ec6b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:56.116864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:56.117947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.118500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.118984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.120445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=9ee3e17b3ac64517937fc782b2e0b10c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:56.121266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:56.123422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf848ca8-bf4a-4baf-a671-712598e57055 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:56.135473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 152/294] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:55 2022] GET /identity/v3/role_assignments?user.id=9ee3e17b3ac64517937fc782b2e0b10c&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=8a09ae9946bc4e158d491383af34a89f => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:56.644500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5f6f7-de3f-48b6-ba3e-cfc998e1e513 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.644848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5f6f7-de3f-48b6-ba3e-cfc998e1e513 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.645180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5f6f7-de3f-48b6-ba3e-cfc998e1e513 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.646803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 143/295] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:56.651959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9e601-3a54-416e-a9a1-4882be59ea1b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.652393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9e601-3a54-416e-a9a1-4882be59ea1b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.652558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9e601-3a54-416e-a9a1-4882be59ea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.673062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f3f9e601-3a54-416e-a9a1-4882be59ea1b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:56.674255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f3f9e601-3a54-416e-a9a1-4882be59ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:56.757789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 153/296] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:56.843417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366fa2b-0467-45a1-be91-359104157095 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.844346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366fa2b-0467-45a1-be91-359104157095 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.844834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366fa2b-0467-45a1-be91-359104157095 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.846419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 144/297] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:56.854585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.854585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.854585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.871107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:56.872287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:56.913874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d72fc-54df-42e5-a488-bee07c92cb8f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.914175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d72fc-54df-42e5-a488-bee07c92cb8f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.914353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d72fc-54df-42e5-a488-bee07c92cb8f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.915712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 145/298] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:56.924306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:56.925202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:56.926465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:56.927043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:56.950581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 154/299] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:56.952867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:56.954209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:57.024755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 146/300] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:55:57.048794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.192697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd23c5-5692-4ba9-a4fb-3229bf007551 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.228956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.229403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.229683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.229951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.230971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.242166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:57.247224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:57.248108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:55:57.249182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 155/301] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:57.254449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.309174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:57.348810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:57.352195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19a5d-5558-4b2d-b50a-49f1a0e6dccc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.383687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34143-3554-4003-9858-1ea2aa800e82 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.391060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.391699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.391975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.392256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.402922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:57.408304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:57.409183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:57.409966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 147/302] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:57.410659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.426989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.427588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.427866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.428136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.428801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:57.429097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:57.430441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:57.439384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 156/303] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:57.440201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.515518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593b7cd-7a33-4a8e-8e35-39a6c7161e10 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.545785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4b2a-deed-479e-af00-85743d84a3d4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.548649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:57.550346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.550994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.551278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.551553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.552666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:57.554131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:57.555744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 148/304] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:57.564105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.578759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.579361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.579632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.579901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.580556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:57.580852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:57.582197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8736f54-b09d-4d02-9f05-c6345580c702 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:57.584112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:57.595656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 157/305] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:57.595656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a6648-494b-464e-9893-8c91b105f83d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.595656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a6648-494b-464e-9893-8c91b105f83d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.595656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a6648-494b-464e-9893-8c91b105f83d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.595656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 158/306] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:55:57.595656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a6648-494b-464e-9893-8c91b105f83d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.655837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc0be-2839-4487-9b83-32cc15832ea9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.688683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.689301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.689574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.689845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] PATH_INFO: `/v3/users/ec587c55b10f4d74bb39cdbf31eb89dd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.690980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] RBAC: Authorizing `identity:get_user(user_id=ec587c55b10f4d74bb39cdbf31eb89dd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:57.692415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:57.694124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 149/307] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/users/ec587c55b10f4d74bb39cdbf31eb89dd => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:57.694742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.699300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a6648-494b-464e-9893-8c91b105f83d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.732267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afa0db-736a-48b6-b89f-2212e56c0b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.732862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afa0db-736a-48b6-b89f-2212e56c0b68 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.733139 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afa0db-736a-48b6-b89f-2212e56c0b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.733410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afa0db-736a-48b6-b89f-2212e56c0b68 None admin] PATH_INFO: `/v3/groups` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.734527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afa0db-736a-48b6-b89f-2212e56c0b68 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:57.735975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afa0db-736a-48b6-b89f-2212e56c0b68 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:57.789642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:57.795166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b0eb6-2685-4f7c-8214-9e7b73944544 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:57.819102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 159/308] 10.222.0.46 () {60 vars in 1134 bytes} [Wed Sep 7 18:55:57 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:55:57.820485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.820764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.827375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.837099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:57.837756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:57.838098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:57.838413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] PATH_INFO: `/v3/users/f0a9e70d749f44ecab09045d97448884` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:57.840944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:55:57.842176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:55:57.857402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] RBAC: Authorizing `identity:get_user(user_id=f0a9e70d749f44ecab09045d97448884)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:57.859552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:57.861329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 150/309] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/users/f0a9e70d749f44ecab09045d97448884 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:57.862057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.867537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:57.930933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 160/310] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:55:57.934575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:57.988817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d4f5-7204-4804-af15-2f7392d6120b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:58.031451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:58.032109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:58.032383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:58.032650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] PATH_INFO: `/v3/system/users/ec587c55b10f4d74bb39cdbf31eb89dd/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:58.034072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ec587c55b10f4d74bb39cdbf31eb89dd, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:58.035521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:58.064454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828b5a-bcc6-41ef-af57-78e5bbef28f6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:58.106840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:58.107855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:58.108324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:58.108854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:58.109777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] RBAC: Adding query filter params (role.id=f448f4dca91e4f1982f116ee3ff052b6, user.id=f0a9e70d749f44ecab09045d97448884, scope.system=all) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:58.110271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:58.112172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:58.117601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 151/311] 10.222.0.46 () {58 vars in 1334 bytes} [Wed Sep 7 18:55:57 2022] PUT /identity/v3/system/users/ec587c55b10f4d74bb39cdbf31eb89dd/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:55:58.121675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 161/312] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Sep 7 18:55:57 2022] GET /identity/v3/role_assignments?user.id=f0a9e70d749f44ecab09045d97448884&scope.system=all&role.id=f448f4dca91e4f1982f116ee3ff052b6 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:58.213729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:58.239625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:58.337859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c08db-37a2-4b4c-9b1b-464b8ea6704e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:58.372581 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:58.373219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:58.373495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:58.373769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:58.384889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:55:58.392065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:55:58.392943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:55:58.393731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:55:58.394323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 152/313] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:55:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:55:58.401776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:58.427254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:58.509587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49eb3a-1824-45ae-bf35-904cfddd0a5c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:58.544778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:58.545729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:58.546984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:58.547447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:58.548300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:58.548904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:58.550513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:58.559417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 162/314] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:55:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:58.567910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:58.649632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:58.672837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71987fa7-df40-43b7-93ad-3a909fa86330 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:58.709049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:58.709686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:58.709965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:58.710237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] PATH_INFO: `/v3/users/48044362b1ab4a76acbdd5d1fc56b9ae` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:58.711468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] RBAC: Authorizing `identity:get_user(user_id=48044362b1ab4a76acbdd5d1fc56b9ae)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:58.712930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:58.714519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 153/315] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:55:58 2022] GET /identity/v3/users/48044362b1ab4a76acbdd5d1fc56b9ae => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:58.720343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:58.805121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:58.824453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e84d-01d8-48a7-ae53-be659ca78799 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:58.860885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:58.861500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:58.861777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:58.862053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:58.935487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:58.937377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:59.019015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:55:59.045591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 163/316] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:55:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:55:59.051795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:59.167133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f553ab-2f3b-4864-8389-3dee3c329f5d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:59.203088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0de85a-8293-4e55-9c6f-17d381d3cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:59.203635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0de85a-8293-4e55-9c6f-17d381d3cc4a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:59.203908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0de85a-8293-4e55-9c6f-17d381d3cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:59.204176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0de85a-8293-4e55-9c6f-17d381d3cc4a None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:59.205284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0de85a-8293-4e55-9c6f-17d381d3cc4a None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:59.206805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0de85a-8293-4e55-9c6f-17d381d3cc4a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:59.208729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 154/317] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:55:59 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:55:59.214698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:55:59.316866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9d047-3996-4fb7-a148-d82c4fc0a123 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:55:59.352778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:55:59.353713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:55:59.354204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:55:59.354691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:55:59.355604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] RBAC: Adding query filter params (role.id=f448f4dca91e4f1982f116ee3ff052b6, scope.project.id=19d2ad19ff1b486589350b08b8845909, user.id=48044362b1ab4a76acbdd5d1fc56b9ae) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:55:59.356198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:55:59.357880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d2964-b369-48d7-bd51-fd04a881ac37 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:55:59.364213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 164/318] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:55:59 2022] GET /identity/v3/role_assignments?user.id=48044362b1ab4a76acbdd5d1fc56b9ae&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=f448f4dca91e4f1982f116ee3ff052b6 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:00.112823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a688ad-edd5-465f-b389-e8883688468e None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.113346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a688ad-edd5-465f-b389-e8883688468e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.113346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a688ad-edd5-465f-b389-e8883688468e None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.114709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 155/319] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:00.120351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1c178-1cd5-40e9-8dee-8af57fe7d7a1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.120747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1c178-1cd5-40e9-8dee-8af57fe7d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.120901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1c178-1cd5-40e9-8dee-8af57fe7d7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.139093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bbb1c178-1cd5-40e9-8dee-8af57fe7d7a1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:00.140804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bbb1c178-1cd5-40e9-8dee-8af57fe7d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:00.183257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:00.208473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 165/320] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:00.403893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9594a27-389e-4e9d-82b2-4e3b3c9dc86a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.404083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9594a27-389e-4e9d-82b2-4e3b3c9dc86a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.404346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9594a27-389e-4e9d-82b2-4e3b3c9dc86a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.406218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 156/321] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:00.412444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.412785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.413064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.417308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af973f3a-59c7-453e-9edb-96a5ab42de52 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.417597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af973f3a-59c7-453e-9edb-96a5ab42de52 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.417878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af973f3a-59c7-453e-9edb-96a5ab42de52 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.419246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 157/322] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:00.424894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.425175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.425450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.432461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:00.433686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:00.444621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:00.447686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:00.458954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:00.506901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 166/323] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:00.507759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:00.524592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 158/324] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:00.607970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eefd8-9c7a-4e11-80e1-ee3477883540 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:00.616121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:00.642830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:00.643458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.643738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.644010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.645107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:00.646538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:00.650586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 167/325] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:00.653642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:00.756562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602991-12f4-4843-88ce-7a04f717b839 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:00.772955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:00.794141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:00.794861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.795127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.795398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] PATH_INFO: `/v3/groups` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.796545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:00.797997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:00.871099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b96e7-f136-4397-bbcb-17c5dc7fd22c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:00.881229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 159/326] 10.222.0.46 () {60 vars in 1134 bytes} [Wed Sep 7 18:56:00 2022] POST /identity/v3/groups => generated 226 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:00.882088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:00.892311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:00.909405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:00.910056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:00.910337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:00.910641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:00.925511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:00.931059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:00.932194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:56:00.933179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:00.934933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 168/327] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:00.940643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:00.981004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:01.012121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea23229-e2e4-4710-953a-ab178f1c826c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.051796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.052661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.052939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.054600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.056697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97a604-e4da-4262-b149-071a658b1208 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.067264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:01.072766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:01.073646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:01.074476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 160/328] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:01.080649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:01.101268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.101877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.102154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.102433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.103297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:01.103434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:01.104798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:01.113899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 169/329] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:01.119140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:01.157505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:01.209311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57bf20-8947-4a15-8ada-f6f1c20da054 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.243672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:01.252257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98ec7b-a12d-4f53-acd1-490aeb5b67ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.262666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.262849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.263125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.263595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.264277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:01.265132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:01.267503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:01.279177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 161/330] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:01.285045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:01.305774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.306882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.307216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.307534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] PATH_INFO: `/v3/users/f0a9e70d749f44ecab09045d97448884` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.308650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a9e70d749f44ecab09045d97448884)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:01.310248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:01.312472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 170/331] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:56:01 2022] GET /identity/v3/users/f0a9e70d749f44ecab09045d97448884 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:01.319189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:01.415612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4076d-a30d-48fb-90af-1e047a52e610 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.445959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:01.450036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395710e2-e884-4e37-928c-f12964b0e696 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.455950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.456119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.456349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.456626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] PATH_INFO: `/v3/users/ec587c55b10f4d74bb39cdbf31eb89dd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.458394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] RBAC: Authorizing `identity:get_user(user_id=ec587c55b10f4d74bb39cdbf31eb89dd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:01.459324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:01.461230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 162/332] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:56:01 2022] GET /identity/v3/users/ec587c55b10f4d74bb39cdbf31eb89dd => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:01.469093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:01.495554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c12b5-ca71-4d50-8130-40f19f062208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.496862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c12b5-ca71-4d50-8130-40f19f062208 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.497406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c12b5-ca71-4d50-8130-40f19f062208 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.497883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c12b5-ca71-4d50-8130-40f19f062208 None admin] PATH_INFO: `/v3/system/users/f0a9e70d749f44ecab09045d97448884/roles/f448f4dca91e4f1982f116ee3ff052b6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.500228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c12b5-ca71-4d50-8130-40f19f062208 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f0a9e70d749f44ecab09045d97448884, role_id=f448f4dca91e4f1982f116ee3ff052b6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:01.501986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c12b5-ca71-4d50-8130-40f19f062208 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:01.573461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 171/333] 10.222.0.46 () {58 vars in 1334 bytes} [Wed Sep 7 18:56:01 2022] PUT /identity/v3/system/users/f0a9e70d749f44ecab09045d97448884/roles/f448f4dca91e4f1982f116ee3ff052b6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 18:56:01.589062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea04414-c782-40f9-8a63-96d10cbf9f09 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:01.630974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:01.632061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:01.632540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:01.633113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:01.634105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] RBAC: Adding query filter params (role.id=8a09ae9946bc4e158d491383af34a89f, user.id=ec587c55b10f4d74bb39cdbf31eb89dd, scope.system=all) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:01.634698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:01.636439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c2462-9507-450a-be78-4bfc1d3eb956 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:01.651958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 163/334] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Sep 7 18:56:01 2022] GET /identity/v3/role_assignments?user.id=ec587c55b10f4d74bb39cdbf31eb89dd&scope.system=all&role.id=8a09ae9946bc4e158d491383af34a89f => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:03.559088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d70d3-6bb3-4fec-9c0c-3a737f1d0215 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:03.559600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d70d3-6bb3-4fec-9c0c-3a737f1d0215 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:03.559600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d70d3-6bb3-4fec-9c0c-3a737f1d0215 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:03.561190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 172/335] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:03.579337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979059fd-96af-42f4-946c-0a4949f9b130 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:03.579834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979059fd-96af-42f4-946c-0a4949f9b130 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:03.579908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979059fd-96af-42f4-946c-0a4949f9b130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:03.607566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-979059fd-96af-42f4-946c-0a4949f9b130 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:03.609432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-979059fd-96af-42f4-946c-0a4949f9b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:03.667035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:03.692921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 164/336] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:04.080283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bba38-550c-42ae-8498-523fd69b777c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.080666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bba38-550c-42ae-8498-523fd69b777c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.080840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bba38-550c-42ae-8498-523fd69b777c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.082428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 173/337] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:04.088850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.089442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.089908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.112457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:04.114187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:04.188712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7cbab-fe85-47bb-be6e-11538706b80c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.189031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7cbab-fe85-47bb-be6e-11538706b80c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.189273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7cbab-fe85-47bb-be6e-11538706b80c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.190879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 174/338] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:04.197822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.198098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.198371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.208093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 165/339] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:04.235555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:04.236778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:04.242280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:04.284220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:04.365067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 175/340] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:04.398341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1652d-74bf-4de8-b499-55c1ab588a86 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:04.419417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:04.450014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6002-257b-49af-ae13-55ef200e1a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:04.450682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6002-257b-49af-ae13-55ef200e1a43 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.450963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6002-257b-49af-ae13-55ef200e1a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.451235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6002-257b-49af-ae13-55ef200e1a43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.470038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6002-257b-49af-ae13-55ef200e1a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-500e6002-257b-49af-ae13-55ef200e1a43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:04.474988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:04.475835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:04.476644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 166/341] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:04.483427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:04.579750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1bfef-cdd4-4eb5-9bfa-4b9434e13204 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.579750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1bfef-cdd4-4eb5-9bfa-4b9434e13204 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.579750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1bfef-cdd4-4eb5-9bfa-4b9434e13204 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.585551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 167/342] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:56:04.587734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.588201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.588635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.625432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:04.629531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910f7f7-fbd1-4986-8888-0adfd5b9d0ce None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:04.631418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:04.714362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:04.714998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.715275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.715549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.716217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:04.716520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:04.718368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:04.736078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 176/343] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:04.736998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:04.760319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 168/344] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:04.761255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:04.787158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:04.895179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda3c7-ff6d-4c13-9b62-e66de8248cbb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:04.896786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:04.933298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:04.934721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:04.935004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:04.935276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:04.940572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff3e38-7d66-46c7-9ae1-3521782800ad None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.009318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.009964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.010243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.010517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.025569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.027580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.051531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:05.056651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:05.057548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 177/345] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:05.059090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.190585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:05.192626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 169/346] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:05.193625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.196338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:05.214256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c4e2-df86-43b3-8b40-8ecf19a338f6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.270528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.271251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.271520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.271796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.285113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:05.291480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:05.292395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:05.293234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:05.296589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 178/347] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:05.297635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.326457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa52f-b30c-49ca-8c64-7fc481c2e0f2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.368553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.369491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.369967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.370511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.371412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:05.372041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.373691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.383153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 170/348] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:05.384618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.428025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545c106-ac04-400f-8af7-83d382f84602 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.461677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:05.469812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.470913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.471549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.472041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.482323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:05.486844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:05.492669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:05.493628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:05.494412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 179/349] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:05.495181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.536567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834bfd-55f8-4e09-b9dc-786ce3cbf6e2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.572419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.573045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.573317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.573587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.575407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.577404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.579160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 171/350] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:05.579927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.632775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167e2ef-33d5-433b-94ab-8f77f3397b52 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.677942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.678582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.678848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.679124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.679784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:05.680082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.681447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.690194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 180/351] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:05.693247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.693391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9642d-19da-4c70-8787-a611e087301a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.725943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:05.733989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.734672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.734947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.735256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] PATH_INFO: `/v3/users/f0a9e70d749f44ecab09045d97448884` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.736468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a9e70d749f44ecab09045d97448884)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.737909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.739518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 172/352] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/users/f0a9e70d749f44ecab09045d97448884 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:05.740274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.765403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:05.812630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5be9b-2bda-428c-8718-3f02eecdfda9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.858126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.859486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.859486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.859486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.859964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:05.860267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.861636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288942-7a5b-42e5-bc4b-71ed4ebd1a26 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.877213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 181/353] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:05.878361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c04db-e90b-4d36-a3fd-8a24a615fa28 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.878682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c04db-e90b-4d36-a3fd-8a24a615fa28 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.878961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c04db-e90b-4d36-a3fd-8a24a615fa28 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.879795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56902e-548d-4ce3-870b-e9b13eba4467 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:05.880244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 182/354] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:05.880947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c04db-e90b-4d36-a3fd-8a24a615fa28 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:05.923462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:05.924323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:05.924611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:05.924887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:05.932290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:05.934172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:05.935833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 173/355] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:05.937241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.006577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.019258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c04db-e90b-4d36-a3fd-8a24a615fa28 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.046949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.054712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8b25d-c85c-49ea-8f03-c4621da7bc25 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.059081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7880a71-839d-414f-b58e-34655ab9df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.059239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7880a71-839d-414f-b58e-34655ab9df4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.059465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7880a71-839d-414f-b58e-34655ab9df4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.059734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7880a71-839d-414f-b58e-34655ab9df4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.105707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.106782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.106782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.107010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.108374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] RBAC: Adding query filter params (role.id=f448f4dca91e4f1982f116ee3ff052b6, user.id=f0a9e70d749f44ecab09045d97448884, scope.system=all) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:06.108756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:06.110353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:06.126501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 174/356] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/role_assignments?user.id=f0a9e70d749f44ecab09045d97448884&scope.system=all&role.id=f448f4dca91e4f1982f116ee3ff052b6 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:06.127563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.173192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7880a71-839d-414f-b58e-34655ab9df4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:06.174853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7880a71-839d-414f-b58e-34655ab9df4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:06.271255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce630374-eba0-4883-9ba9-4cf94570e280 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.302599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.314303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 183/357] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:06.318948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.318948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.318948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.322067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1704b-dac0-40d6-a7d2-1296044e1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.322762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1704b-dac0-40d6-a7d2-1296044e1e8e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.323346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1704b-dac0-40d6-a7d2-1296044e1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.323615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1704b-dac0-40d6-a7d2-1296044e1e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.335722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:06.337241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:06.357201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.418375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1704b-dac0-40d6-a7d2-1296044e1e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:06.419873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1704b-dac0-40d6-a7d2-1296044e1e8e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:06.421949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 184/358] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:06.422975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.548200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 175/359] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:06.549408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711200-bf1e-4dc3-b316-e69d784a19ab None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.549687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711200-bf1e-4dc3-b316-e69d784a19ab None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.549956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711200-bf1e-4dc3-b316-e69d784a19ab None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.555654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 176/360] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:06.555654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711200-bf1e-4dc3-b316-e69d784a19ab None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.563513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d1852-614d-442e-a8ff-43a00cb6345a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.582789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.607642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.609144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.609687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.609974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.611864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=20b1e025ed514544abb2932a8dace90d) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:06.612122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:06.613558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc550f-c91c-4d02-b006-64adea342356 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:06.623097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 185/361] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=8a09ae9946bc4e158d491383af34a89f => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:06.625098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.625661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.625661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.648132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:06.649473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:06.665266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.675657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711200-bf1e-4dc3-b316-e69d784a19ab None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.728521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.729173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.729485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.729746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.730969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:06.733793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:06.735577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 177/362] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:06.736377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.737348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 186/363] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:06.738319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.853920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:06.867697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec446-9a49-42dc-a1c1-ca415f6fbee8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.879853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa665c-9039-44c0-9c3f-736202d86c9c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:06.917142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.918211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.918850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.919317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.921241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:06.923168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:06.925330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 178/364] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:06.926342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.927618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:06.927618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:06.927912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:06.928140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:06.940170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:06.946364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:06.947328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:06.948127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 187/365] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:06.948801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:06.966502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.046612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c842-765f-43c3-8af2-6361d5505203 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.084301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75752-deba-4cde-acba-17c1f4ce72c0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.094059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.095373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.095654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.095928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.097212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.099451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.102329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 179/366] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:07.103204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.137997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.194183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.194183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.194183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.194472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.194883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.194883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.194883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 188/367] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:06 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:07.194883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.251529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6b918-0dd7-45ec-9668-64443dd3921c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.252118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.281211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1fcb8-e33d-4b72-b3a9-7ebe70cb3d10 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.300924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.301543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.301818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.302087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.302789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:07.303091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.304460 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.316244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 180/368] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:07.316891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.328481 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.329119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.329394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.329664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.343009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:07.347872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:07.348794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:07.349634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 189/369] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:07.350389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.443502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.458738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881be60b-6166-4c00-a206-f41cac640ce3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.479121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780012ee-ed4e-4cb3-86bf-c461cf5a91a0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.497212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.497594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.497971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.498255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.500710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=20b1e025ed514544abb2932a8dace90d) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:07.500710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.501819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.508292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 181/370] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:07.509011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.517625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.518760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.519249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.519737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.521113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=19d2ad19ff1b486589350b08b8845909) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:07.522669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.524263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.531184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.538617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 190/371] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=8a09ae9946bc4e158d491383af34a89f => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:07.542593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.659999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dd4f9-81d1-44dc-8190-a841d0f45772 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.702599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.703145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.703567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.703843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.704532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:07.704831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.706208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.715823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 191/372] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:07.727006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:07.745160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21080b-7cf6-418a-a1d7-92dde753c9a9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.801251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.818984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.819605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.819884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.820162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.843572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc4d25-78f8-4351-b7bc-afe4c3b92d6a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:07.867695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:07.883737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:07.884363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:07.884647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:07.884899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:07.923844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.925496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:07.965461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:07.968074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.040795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:08.041696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 182/373] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:08.047373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:08.098393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 192/374] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:56:08.103721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:08.117533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:08.206374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fd1e1-1c48-4424-a3a9-d334cfd5babf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:08.281380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:08.282019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:08.282654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:08.282936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:08.284796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:08.286229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.288811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 183/375] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:08 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:08.295107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:08.303137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773da016-8f0b-42d5-a148-0a0382a69165 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:08.376658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:08.377409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:08.377692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:08.377965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] PATH_INFO: `/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:08.380689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=0eeffadf972f483fb4e32d2c25b5e94c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:08.384404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.384404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 193/376] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:08 2022] GET /identity/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c => generated 344 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:08.390160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:08.467776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:08.479561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b18e99-1395-4cc8-9125-f685629bfe9a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:08.503680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:08.521992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:08.524081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:08.524081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:08.524081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:08.524982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:08.526600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.528859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 184/377] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:08 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:08.534832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:08.643603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f0d2-cfc6-487b-ae7e-d0cad1ac2ec2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:08.723708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:08.726017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:08.729060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:08.729507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] PATH_INFO: `/v3/groups/19e9f53161ac408db9599d801c09f5b4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:08.741967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] RBAC: Authorizing `identity:get_group(group_id=19e9f53161ac408db9599d801c09f5b4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:08.743803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.746078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 194/378] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:08 2022] GET /identity/v3/groups/19e9f53161ac408db9599d801c09f5b4 => generated 229 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:08.752181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:08.753903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e89d4-d0f5-439d-8098-52f16d584e4f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:08.791986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:08.792599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:08.792877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:08.793151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:08.793930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=19d2ad19ff1b486589350b08b8845909) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:08.794187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:08.795954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202e52f-7d99-42fd-8614-5a07a7319d59 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.805080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 185/379] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:08 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:08.897892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:08.906146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96229d9-746f-4d48-ac6b-cd5c42e4a9bc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:08.951465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:08.952102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:08.952382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:08.952669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:08.953595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] RBAC: Adding query filter params (group.id=19e9f53161ac408db9599d801c09f5b4, role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0eeffadf972f483fb4e32d2c25b5e94c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:08.953930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:08.955430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e823f-e177-44ef-a039-203ee8dde16d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:08.967356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 195/380] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:08 2022] GET /identity/v3/role_assignments?group.id=19e9f53161ac408db9599d801c09f5b4&scope.project.id=0eeffadf972f483fb4e32d2c25b5e94c&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:09.191087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3248-9ca9-424e-8cb7-e5fd8d156f05 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:09.191461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3248-9ca9-424e-8cb7-e5fd8d156f05 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:09.191666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3248-9ca9-424e-8cb7-e5fd8d156f05 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:09.193874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 186/381] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:09.199119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:09.199391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:09.199662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:09.225177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:09.226397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:09.324647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 196/382] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:09.749738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3248-9ca9-424e-8cb7-e5fd8d156f05 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:09.821066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:09.880621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3248-9ca9-424e-8cb7-e5fd8d156f05 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:09.915653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:09.916289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:09.916566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:09.916842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:09.927906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:09.933240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:09.934149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:09.935009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 187/383] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:09.942585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:10.059549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:10.077241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b525e9-258d-4c64-9bbc-fa0dec0f7120 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:10.124290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:10.124907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.125188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.125461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.126133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:10.126434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:10.127842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e045ed0-2524-4886-ad81-5480c06d5308 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:10.142230 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 197/384] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:10.147764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:10.285303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68b919-04e7-4829-be07-a02a847b9615 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:10.290532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b5c2d-3cf8-483c-b1e3-f709bb8fa3de None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.291087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b5c2d-3cf8-483c-b1e3-f709bb8fa3de None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.291582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b5c2d-3cf8-483c-b1e3-f709bb8fa3de None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.293349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 198/385] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:10.298518 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.299097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.299589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.307699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:10.333572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:10.336729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:10.354667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:10.355634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.356130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.358251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.360470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:10.362272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:10.366141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 188/386] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:10 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:10.374102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:10.447144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 199/387] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:10.554447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b14b9-bf5e-48b8-ae6c-e41a196986d0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:10.599950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b14199-0052-4269-95ed-5563232a1121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:10.600988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b14199-0052-4269-95ed-5563232a1121 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.601467 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b14199-0052-4269-95ed-5563232a1121 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.601988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b14199-0052-4269-95ed-5563232a1121 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.607284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b14199-0052-4269-95ed-5563232a1121 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:10.609450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b14199-0052-4269-95ed-5563232a1121 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:10.611433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 189/388] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:10 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:10.617507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:10.715349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe10cc5-8337-4034-81e5-a18989a488fb None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.716002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe10cc5-8337-4034-81e5-a18989a488fb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.716504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe10cc5-8337-4034-81e5-a18989a488fb None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.718438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 190/389] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:56:10.724299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.724807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.726040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.750609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:10.764937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:10.766155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:10.779179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba09d8e-943f-4913-8f4b-8eef443e69b9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:10.816896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:10.818504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:10.819271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:10.819620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:10.819967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d/groups/921ccc6b9791486996cf96258605d1a5/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:10.822753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] RBAC: Authorizing `identity:create_grant(project_id=20b1e025ed514544abb2932a8dace90d, group_id=921ccc6b9791486996cf96258605d1a5, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:10.826500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:10.844266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 191/390] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:10.845753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:10.945385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 200/391] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Sep 7 18:56:10 2022] PUT /identity/v3/projects/20b1e025ed514544abb2932a8dace90d/groups/921ccc6b9791486996cf96258605d1a5/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:10.966436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1842e-4fd1-44c1-a057-8345cbccfaf0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.010852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.011503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.012689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.012689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.025160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:11.030206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:11.031151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:11.032380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 192/392] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:11.039723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.186466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.212799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fb878-d137-42c1-9565-cf7033d41c32 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.252459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:11.257696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.258635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.259059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.259500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.260312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:11.260759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:11.262987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:11.272052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 201/393] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:11.277878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.280774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:11.327134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaba86f-8157-4a1a-8678-c06a73f9e4de None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.364307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.364632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.364860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.365134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.375858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:11.380900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:11.381800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:11.382618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 193/394] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:11.388028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.408235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74890418-bf6d-497d-a3d4-acf55f205e6d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.466066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.466780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.467645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.467852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.470684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:11.472164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:11.474202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 202/395] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:11.481270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.593048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740eaa3c-ff91-48c8-922f-bfe144b56880 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.595413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:11.610948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:11.629214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b62c9-369c-4858-b4f2-47f212280aac None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.635379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.636353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.636848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.637332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.638206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:11.639447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:11.641042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3934-d266-4fca-9ab6-437649fa1fc7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:11.652216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 194/396] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:11.653996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba60e4-5647-43cd-b3ae-7855d25cea93 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.655230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba60e4-5647-43cd-b3ae-7855d25cea93 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.655748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba60e4-5647-43cd-b3ae-7855d25cea93 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.657692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 195/397] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:11.658952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba60e4-5647-43cd-b3ae-7855d25cea93 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.667800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a23f96-6ea6-49af-9502-ff6b4306a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.668440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a23f96-6ea6-49af-9502-ff6b4306a773 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.668721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a23f96-6ea6-49af-9502-ff6b4306a773 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.668999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a23f96-6ea6-49af-9502-ff6b4306a773 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.670749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a23f96-6ea6-49af-9502-ff6b4306a773 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:11.672284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a23f96-6ea6-49af-9502-ff6b4306a773 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:11.673948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 203/398] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:11.679816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b89f5-e9e6-4d2a-b70a-11dbf047e3c1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.680437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b89f5-e9e6-4d2a-b70a-11dbf047e3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.680437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b89f5-e9e6-4d2a-b70a-11dbf047e3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.699009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1d2b89f5-e9e6-4d2a-b70a-11dbf047e3c1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:11.700215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1d2b89f5-e9e6-4d2a-b70a-11dbf047e3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:11.775858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 204/399] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:11.777371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facbe28-4554-4bfd-8d4c-cb3ed9327f32 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.777828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facbe28-4554-4bfd-8d4c-cb3ed9327f32 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.778297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facbe28-4554-4bfd-8d4c-cb3ed9327f32 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.779948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 205/400] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:11.780784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facbe28-4554-4bfd-8d4c-cb3ed9327f32 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.785676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba60e4-5647-43cd-b3ae-7855d25cea93 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.829366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdd64d-c896-47c8-8c41-319bdf4ab011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.830035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdd64d-c896-47c8-8c41-319bdf4ab011 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.830316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdd64d-c896-47c8-8c41-319bdf4ab011 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.830615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdd64d-c896-47c8-8c41-319bdf4ab011 None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.831779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdd64d-c896-47c8-8c41-319bdf4ab011 None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:11.833336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdd64d-c896-47c8-8c41-319bdf4ab011 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:11.840349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 196/401] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:11.840349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.840349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.840349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.861070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:11.862314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:11.877619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:11.898936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:11.925052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facbe28-4554-4bfd-8d4c-cb3ed9327f32 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:11.962894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 197/402] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:11.963813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:11.967902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:11.969193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:11.969675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:11.970182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d/groups/921ccc6b9791486996cf96258605d1a5/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:11.973674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] RBAC: Authorizing `identity:create_grant(project_id=20b1e025ed514544abb2932a8dace90d, group_id=921ccc6b9791486996cf96258605d1a5, role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:11.975441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:12.057348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 206/403] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Sep 7 18:56:11 2022] PUT /identity/v3/projects/20b1e025ed514544abb2932a8dace90d/groups/921ccc6b9791486996cf96258605d1a5/roles/b6752aeca4994d6292f5216c4d87e805 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:12.058530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.123884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71394179-2e49-4afd-9d2c-5f9f39868493 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.182732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.183864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.184357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.184828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.186197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:12.188010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:12.190320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 198/404] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:11 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:12.196614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.223261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131225-ccee-4b88-85f6-8d9f3fee8721 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.260411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:12.262308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:12.271475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.276653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.276653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.276653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.290895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:12.299030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:12.299933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:12.300729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 207/405] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:12.307264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.364271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6597098-0b90-4b43-a9bd-f7ffca3f98ab None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.409656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.410700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.411154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.411840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909/groups/921ccc6b9791486996cf96258605d1a5/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.413967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] RBAC: Authorizing `identity:create_grant(project_id=19d2ad19ff1b486589350b08b8845909, group_id=921ccc6b9791486996cf96258605d1a5, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:12.415722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:12.442472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e5c0d-a004-4b27-afc4-8b0be1339c78 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.478257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.478916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.479199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.479479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.480156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:12.480461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:12.481836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:12.496673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 208/406] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:12.496673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.527731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 199/407] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Sep 7 18:56:12 2022] PUT /identity/v3/projects/19d2ad19ff1b486589350b08b8845909/groups/921ccc6b9791486996cf96258605d1a5/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 18:56:12.535433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.599063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:12.706541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d82d1f-924e-4d3d-9b03-841f0b6ae875 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.723639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:12.744106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885bbc-8cc3-4536-ad0c-5d3d4884dc65 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.754379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.755124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.755474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.755831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.768498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:12.773949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:12.774956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:12.775803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 209/408] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:12.781609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.813261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.814483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.816264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.817487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] PATH_INFO: `/v3/groups/19e9f53161ac408db9599d801c09f5b4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.820480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] RBAC: Authorizing `identity:get_group(group_id=19e9f53161ac408db9599d801c09f5b4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:12.822579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:12.826844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 200/409] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:12 2022] GET /identity/v3/groups/19e9f53161ac408db9599d801c09f5b4 => generated 229 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:12.832445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:12.914146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d07758-bb20-4c24-9cb4-3d5287131260 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:12.938898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:12.985439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:12.986064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:12.986334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:12.986507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:12.987191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:12.987489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:12.988852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:13.005299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 210/410] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:13.014933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:13.040563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b784724-b2f5-431d-9d5e-eca3dcfc4acb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:13.097499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:13.098335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.098660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.099278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] PATH_INFO: `/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.100721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] RBAC: Authorizing `identity:get_project(project_id=0eeffadf972f483fb4e32d2c25b5e94c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:13.102337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:13.104328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 201/411] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:12 2022] GET /identity/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c => generated 344 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:13.110741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:13.151813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c22c8-db71-44b6-adf4-7962b82ea9c9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:13.162918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:13.227576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:13.228212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.228486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.228758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.231762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:13.235200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:13.236006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 211/412] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:13 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:13.244581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:13.302776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efb51-5100-4596-9c25-e0f6aa6bbae9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:13.327449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:13.369213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d987437-a8b2-40ad-9af2-f6bfc90704b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:13.369885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d987437-a8b2-40ad-9af2-f6bfc90704b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.370166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d987437-a8b2-40ad-9af2-f6bfc90704b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.370446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d987437-a8b2-40ad-9af2-f6bfc90704b6 None admin] PATH_INFO: `/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c/groups/19e9f53161ac408db9599d801c09f5b4/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.375834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d987437-a8b2-40ad-9af2-f6bfc90704b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0eeffadf972f483fb4e32d2c25b5e94c, group_id=19e9f53161ac408db9599d801c09f5b4, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:13.377287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d987437-a8b2-40ad-9af2-f6bfc90704b6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:13.393848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb98ea3-4a64-4494-9540-b540949c539d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:13.436623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:13.437593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.438037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.438477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.439844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:13.441769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:13.444021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 212/413] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:13 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:13.449851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:13.458255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 202/414] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Sep 7 18:56:13 2022] PUT /identity/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c/groups/19e9f53161ac408db9599d801c09f5b4/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 18:56:13.531763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d175d4-55cc-4933-99e7-e0c1137d11a3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.532077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d175d4-55cc-4933-99e7-e0c1137d11a3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.532362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d175d4-55cc-4933-99e7-e0c1137d11a3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.535802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 203/415] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:13.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.541498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.541968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.561175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:13.562752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:13.573069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728642b-ea91-4ddc-b42b-e0b7db16d457 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:13.585492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:13.644313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:13.645414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:13.645900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:13.646389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909/groups/921ccc6b9791486996cf96258605d1a5/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:13.651249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=19d2ad19ff1b486589350b08b8845909, group_id=921ccc6b9791486996cf96258605d1a5, role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:13.653934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:13.669118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 204/416] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 18:56:13.725253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 213/417] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Sep 7 18:56:13 2022] PUT /identity/v3/projects/19d2ad19ff1b486589350b08b8845909/groups/921ccc6b9791486996cf96258605d1a5/roles/b6752aeca4994d6292f5216c4d87e805 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:13.975821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:13.992465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:14.087611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3416a-d873-4018-a282-38a2f8554298 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:14.124002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:14.124691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:14.124965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:14.125233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:14.135788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:14.141265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:14.142144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:14.143370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 205/418] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:14.148562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:14.288648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63f58-85d9-493d-90c9-6c263a16d0b3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:14.344211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:14.344838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:14.345117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:14.345388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:14.346064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:14.346367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:14.347761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:14.358019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 214/419] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:14.363465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:14.451459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:14.474016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238087c-3a21-46ca-90bc-f08fe881171f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:14.509435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823e0ac-1e1f-4417-a652-918dd590c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:14.509968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823e0ac-1e1f-4417-a652-918dd590c4dc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:14.510244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823e0ac-1e1f-4417-a652-918dd590c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:14.510516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823e0ac-1e1f-4417-a652-918dd590c4dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:14.583858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823e0ac-1e1f-4417-a652-918dd590c4dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:14.585405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823e0ac-1e1f-4417-a652-918dd590c4dc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:14.662967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:14.695174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 206/420] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:14.700907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:14.735008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:14.769260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49df9-157e-484c-9418-6b942afa1dfe None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:14.769439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49df9-157e-484c-9418-6b942afa1dfe None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:14.769720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49df9-157e-484c-9418-6b942afa1dfe None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:14.771100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 207/421] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:14.776667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:14.776941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:14.777215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:14.812731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcecb5-2c0f-47f9-bfee-65a4d672fd19 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:14.815687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:14.816883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:14.854317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:14.854982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:14.855260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:14.855534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:14.856786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:14.858938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:14.861211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 215/422] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:14 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:14.867188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:14.977003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 208/423] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:14.986916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:14.990535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ff133-4ff6-4294-a434-4f27df3102e4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:15.025709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38463c92-5f9f-439f-93d2-0891f756b5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:15.026352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38463c92-5f9f-439f-93d2-0891f756b5af None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.026659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38463c92-5f9f-439f-93d2-0891f756b5af None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.026937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38463c92-5f9f-439f-93d2-0891f756b5af None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.028156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38463c92-5f9f-439f-93d2-0891f756b5af None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:15.030184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38463c92-5f9f-439f-93d2-0891f756b5af None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:15.031823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 216/424] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:14 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:15.038086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:15.100797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b322c-0fb9-4095-a0bc-f74ecba54afd None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.101080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b322c-0fb9-4095-a0bc-f74ecba54afd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.101513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b322c-0fb9-4095-a0bc-f74ecba54afd None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.103251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 217/425] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:15.113279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.113567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.113846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.148063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:15.150929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:15.227228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169d582-91fd-408e-b240-94d3a2fcea11 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:15.228730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:15.281295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:15.281827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.282108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.282385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.283404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=20b1e025ed514544abb2932a8dace90d) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:15.283714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:15.285560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:15.292487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 209/426] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:15 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=8a09ae9946bc4e158d491383af34a89f => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:15.294089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 218/427] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:15.372436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:15.484481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac407-d0f5-4916-80cc-e2bd36793ab7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:15.522843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:15.523477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.523749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.524017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.535326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:15.540379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:15.541279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:15.542091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 210/428] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:15.547999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:15.642946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:15.665757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:15.671449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8543a-879b-4f47-af5e-c5cf7f351345 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:15.686032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:15.715198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:15.716113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.717125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.717564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.718741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:15.719228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:15.720787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:15.732403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 219/429] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:15.739147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:15.791113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c992069-4385-468b-9b7e-84571d3eeb4b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:15.831421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:15.832058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.832391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.832663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.852498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:15.860429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:15.861302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 18:56:15.862105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:15.867345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 211/430] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:15.875418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:15.903810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10a8f0-9689-4909-9df7-6fb96f8524d4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:15.945705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:15.946443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:15.946844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:15.947188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:15.981513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:15.995657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.027917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5336-fa59-417c-8f90-273283369363 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.042443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.043988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.079962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.081489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.082034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.082509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.083416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:16.084085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.086610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.099181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 212/431] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:56:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:16.105793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.173584 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 220/432] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:56:16.178886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.250060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb76293-f2f0-49df-bda2-b8cc87b186df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.253164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.296375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff41cf3-5225-4458-9ebf-8c52a2637c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.297000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff41cf3-5225-4458-9ebf-8c52a2637c7b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.297340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff41cf3-5225-4458-9ebf-8c52a2637c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.297677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff41cf3-5225-4458-9ebf-8c52a2637c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.297839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78ae02-c188-4a0d-8858-dfe7e2bf5f02 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.317304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.333256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d14e-5f35-4d63-90c6-a37c6f0edc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.334134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d14e-5f35-4d63-90c6-a37c6f0edc79 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.334698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d14e-5f35-4d63-90c6-a37c6f0edc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.334979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d14e-5f35-4d63-90c6-a37c6f0edc79 None admin] PATH_INFO: `/v3/projects/20b1e025ed514544abb2932a8dace90d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.336119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60d14e-5f35-4d63-90c6-a37c6f0edc79 None admin] RBAC: Authorizing `identity:get_project(project_id=20b1e025ed514544abb2932a8dace90d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.337608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60d14e-5f35-4d63-90c6-a37c6f0edc79 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.339374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 221/433] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/projects/20b1e025ed514544abb2932a8dace90d => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:16.340390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db081dc-83c1-4713-9eab-7787b34673a3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.340669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db081dc-83c1-4713-9eab-7787b34673a3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.340942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db081dc-83c1-4713-9eab-7787b34673a3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.346660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 222/434] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:16.347416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db081dc-83c1-4713-9eab-7787b34673a3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.374021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff41cf3-5225-4458-9ebf-8c52a2637c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.375491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff41cf3-5225-4458-9ebf-8c52a2637c7b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.453435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db081dc-83c1-4713-9eab-7787b34673a3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 213/435] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:56:16.481785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.482130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.482494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.486799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.497471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.498360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.499857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.499857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.500470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:16.500720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.501658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:16.502672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.505691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 223/436] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:16.507506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.554089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.578414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 214/437] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:16.579411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.637162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dccf5-5941-439c-89b2-bff190b7a888 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.674180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338575e0-66f3-4d25-afa0-2dbf46f73f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.674838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338575e0-66f3-4d25-afa0-2dbf46f73f8f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.675116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338575e0-66f3-4d25-afa0-2dbf46f73f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.675384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338575e0-66f3-4d25-afa0-2dbf46f73f8f None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.676906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338575e0-66f3-4d25-afa0-2dbf46f73f8f None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.678746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338575e0-66f3-4d25-afa0-2dbf46f73f8f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.680704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 224/438] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:16.685824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fedd6a-ae06-4b6d-9bb4-be8756a1baee None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.685824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fedd6a-ae06-4b6d-9bb4-be8756a1baee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.685824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fedd6a-ae06-4b6d-9bb4-be8756a1baee None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.685824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 225/439] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:16.688440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fedd6a-ae06-4b6d-9bb4-be8756a1baee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.707764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb939197-55dc-41d2-aa79-5a2d888995ad None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.741737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.742334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.742632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.742911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.743632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=20b1e025ed514544abb2932a8dace90d) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:16.743938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.745434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e12ed-1d15-4773-a9c5-72301463e64d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.751946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 215/440] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=20b1e025ed514544abb2932a8dace90d&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:16.753065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.753347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.753618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.758497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.778674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:16.780916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:16.805130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fedd6a-ae06-4b6d-9bb4-be8756a1baee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:16.839302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:16.847887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:16.850728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:16.850728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:16.850728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:16.850728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:16.852539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:16.859012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 226/441] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:16.861379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.883367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 216/442] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:16.884733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:16.989734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf8211-62cd-4528-b37b-9bae6e1a37e8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.019961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264b988-faff-4667-ba51-94d2a5a4553f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.024541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.032270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.032888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.033496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.033811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.034899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=8a09ae9946bc4e158d491383af34a89f, scope.project.id=19d2ad19ff1b486589350b08b8845909) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:17.035298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.036893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.046623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 227/443] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=8a09ae9946bc4e158d491383af34a89f => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:17.066914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.067480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.067766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.068031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.081297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:17.086393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:17.087395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:17.088407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 217/444] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:17.094216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.166295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.214336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.222074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537e4b5-8aed-46e4-911b-c9cf3af0ccfc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.269085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.269704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.269983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.270258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.270973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:17.271269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.272637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.282347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 228/445] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:17.289790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.297996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610d6d1-58a2-4adb-8175-20e4902eb89d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.336476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.337111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.337385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.338105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.351317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:17.356363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:17.357786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:17.358618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 218/446] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:17.359326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.364341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.410653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a688e1-a6ad-4d52-bb9d-e27b707a06e5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.453814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.454418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.454722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.455001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.474352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.482466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc67519-55e5-468e-866e-57610d2c77ec None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.516610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.517207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.517483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.517751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.518406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:17.518736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.520127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.521734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.523208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.528517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 219/447] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:17.534754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.578228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.624679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 229/448] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:17.630204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.635675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63153a-c8dc-4e72-aee6-10bcf3dfd0e0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.669276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.670395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.670695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.670966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.693734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.726651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949aa9e-fb12-453a-8f09-b1f610c7d3b4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.736606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.738074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.759699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.760311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.760583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.760884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] PATH_INFO: `/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.761971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] RBAC: Authorizing `identity:get_project(project_id=0eeffadf972f483fb4e32d2c25b5e94c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.763395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.764996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 230/449] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/projects/0eeffadf972f483fb4e32d2c25b5e94c => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:17.770427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.795795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.840185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 220/450] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:56:17.841189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.874385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174c5e5-1083-4424-9ced-67343e9f465b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.895188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:17.908295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.908919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.909204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.909478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] PATH_INFO: `/v3/groups/19e9f53161ac408db9599d801c09f5b4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.910684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] RBAC: Authorizing `identity:get_group(group_id=19e9f53161ac408db9599d801c09f5b4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.912380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.913999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 231/451] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/groups/19e9f53161ac408db9599d801c09f5b4 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:17.919828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:17.938972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d0122-0d8d-41cc-abf1-010c8b85fb3f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:17.972966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:17.973581 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:17.973851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:17.974123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] PATH_INFO: `/v3/projects/19d2ad19ff1b486589350b08b8845909` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:17.975253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] RBAC: Authorizing `identity:get_project(project_id=19d2ad19ff1b486589350b08b8845909)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:17.976682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:17.978428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 221/452] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/projects/19d2ad19ff1b486589350b08b8845909 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:17.983622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:18.007499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:18.017084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eea9c2-1924-4d55-8122-89d6400ed96c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:18.051979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:18.052631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:18.052909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:18.053182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:18.053912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] RBAC: Adding query filter params (group.id=19e9f53161ac408db9599d801c09f5b4, role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0eeffadf972f483fb4e32d2c25b5e94c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:18.057837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:18.057837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:18.062778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 232/453] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/role_assignments?group.id=19e9f53161ac408db9599d801c09f5b4&scope.project.id=0eeffadf972f483fb4e32d2c25b5e94c&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:18.090149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bfcad-ad7a-4cf4-875a-2b0b86d530c1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:18.115955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:18.128565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b80d8e-c588-4d27-a5c1-e4d99566bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:18.129200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80d8e-c588-4d27-a5c1-e4d99566bc9c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:18.129469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80d8e-c588-4d27-a5c1-e4d99566bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:18.129737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80d8e-c588-4d27-a5c1-e4d99566bc9c None admin] PATH_INFO: `/v3/groups/921ccc6b9791486996cf96258605d1a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:18.130969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b80d8e-c588-4d27-a5c1-e4d99566bc9c None admin] RBAC: Authorizing `identity:get_group(group_id=921ccc6b9791486996cf96258605d1a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:18.132449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b80d8e-c588-4d27-a5c1-e4d99566bc9c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:18.134091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 222/454] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Sep 7 18:56:17 2022] GET /identity/v3/groups/921ccc6b9791486996cf96258605d1a5 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:18.140956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:18.244170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e32cd-42c7-4f81-84d9-5256cae5056d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:18.281335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:18.281942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:18.282221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:18.282495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:18.283248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] RBAC: Adding query filter params (group.id=921ccc6b9791486996cf96258605d1a5, role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=19d2ad19ff1b486589350b08b8845909) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:18.283554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:18.284973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73b975-0f40-48af-be1a-59391e65ba8a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:18.291043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 233/455] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Sep 7 18:56:18 2022] GET /identity/v3/role_assignments?group.id=921ccc6b9791486996cf96258605d1a5&scope.project.id=19d2ad19ff1b486589350b08b8845909&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:21.052953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0dbd0-8a90-4ead-b77c-f4bd03340fbc None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.054006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0dbd0-8a90-4ead-b77c-f4bd03340fbc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.054491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0dbd0-8a90-4ead-b77c-f4bd03340fbc None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.056342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 223/456] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:21.061599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3b7c-736e-4654-9f89-8f13f2ce9500 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.062414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3b7c-736e-4654-9f89-8f13f2ce9500 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.062935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3b7c-736e-4654-9f89-8f13f2ce9500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.087538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3dac3b7c-736e-4654-9f89-8f13f2ce9500 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:21.089185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3dac3b7c-736e-4654-9f89-8f13f2ce9500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:21.112086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:21.181254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 234/457] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:21.274207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c4e4c-be0b-44df-9c9f-99bed41ba101 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.274523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c4e4c-be0b-44df-9c9f-99bed41ba101 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.274795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c4e4c-be0b-44df-9c9f-99bed41ba101 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.276357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 224/458] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:21.288228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d03891-a107-4435-9113-1e659baa993f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.288524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d03891-a107-4435-9113-1e659baa993f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.288805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d03891-a107-4435-9113-1e659baa993f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.315554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-20d03891-a107-4435-9113-1e659baa993f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:21.316674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-20d03891-a107-4435-9113-1e659baa993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:21.405229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 235/459] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:21.565550 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e78aae-b57e-4235-a70b-d57cb32fd660 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.565995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e78aae-b57e-4235-a70b-d57cb32fd660 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.566099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e78aae-b57e-4235-a70b-d57cb32fd660 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.567865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 225/460] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:21.572604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.572892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.573170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.586899 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e78aae-b57e-4235-a70b-d57cb32fd660 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:21.599723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:21.600906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:21.740664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e78aae-b57e-4235-a70b-d57cb32fd660 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:21.768349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 236/461] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:21.821813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:21.822361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:21.822657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:21.822925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:21.843649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:21.855604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:21.857689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:21.884496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:21.895754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:21.896680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:21.897544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:21.897544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:21.897544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:21.904267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 226/462] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:21.905258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.018907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:22.103807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b98e-1c86-444f-9ec5-41271cc5a202 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.156166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033012d-bca8-457a-bd5c-d25244affb30 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.164634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.165241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.165872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.166149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.166827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:22.167178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.168738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfec86-5271-417f-b9b5-4226ae02cf11 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.191950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 227/463] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:22.193540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528151-3aaa-45d7-a4fc-3c5d59b1a1bd None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.193967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528151-3aaa-45d7-a4fc-3c5d59b1a1bd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.194381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528151-3aaa-45d7-a4fc-3c5d59b1a1bd None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.195767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 228/464] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:22.200533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13528151-3aaa-45d7-a4fc-3c5d59b1a1bd None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.219674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.220715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.221006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.221291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.237284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.238971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:22.254852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:22.255974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:22.257023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:22.257023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:22.257023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:22.258378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 237/465] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:22.258378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.319121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13528151-3aaa-45d7-a4fc-3c5d59b1a1bd None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.320601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:22.324413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:22.356123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.357300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.357572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.357839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.367685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f500b1-a42d-4bcc-9d24-199c39b954a6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.369081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.370792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6dddcdc4-1682-4fc9-9523-be9c4825b7cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:22.386673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:22.388438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 229/466] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.394357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.410345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:22.411824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:22.414228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.414864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.415150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.415424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.416102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.417521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.478961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 230/467] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:22.479847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.514581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 238/468] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:56:22 2022] POST /identity/v3/users => generated 246 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:22.514581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.641078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:22.644498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856123a-4b20-4bc8-b8f1-a929e2df4494 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.663542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24481cb-e191-4741-8fea-92cf0de76562 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.665550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:22.687468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.688513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.689122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.689575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.690511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:22.691030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.692680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d26868-b814-4b2e-8fd0-cb0de2c150e4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.709909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 239/469] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:22.710587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.711353 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307417-2646-4b8e-9ba8-0b54d6dc27a0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.711482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.711638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307417-2646-4b8e-9ba8-0b54d6dc27a0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.715698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.715698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.715907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307417-2646-4b8e-9ba8-0b54d6dc27a0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.715981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:22.715981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.716098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 240/470] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:22.716176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab94257-4064-4ce1-b7bc-41ef39b936bf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.716243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc307417-2646-4b8e-9ba8-0b54d6dc27a0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.730202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 231/471] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:22.732222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.732498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.732766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.752076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:22.753954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:22.824937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 232/472] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:22.825799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:22.843827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc307417-2646-4b8e-9ba8-0b54d6dc27a0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.881694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:22.882401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:22.882753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:22.883081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:22.883790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:22.885323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:22.923780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:22.948253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45f08-2b81-4c27-a87e-6fae6507f8bb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:22.951785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 241/473] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:56:22 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:22.952848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:23.034257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.035310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.035845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.036641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.037686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:23.039431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:23.057624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:23.139752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03740e-8578-4fc3-a2f7-f17c84618c0f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:23.174769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.175397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.175636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.175850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.179677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 233/474] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:56:22 2022] POST /identity/v3/users => generated 248 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:23.180745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:23.187793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:23.189353 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:23.207031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:23.208003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:23.208873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:23.208873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:23.208873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:23.208873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 242/475] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:23.219574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:23.327180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b53b8f-f6cb-4482-a0bf-6ee6f24e6337 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:23.346501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:23.362227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:23.362636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689f599-dbaf-4f15-b96f-5b8f7b274d68 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:23.369569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.371168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.371168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.371591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.372566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:23.374335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:23.400215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.400832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.401110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.401385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.402042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:23.402346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:23.404308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:23.417111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 243/476] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:23.422671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:23.428716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.sql.core [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 18:56:23.428716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 18:56:23.428716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [parameters: {'id': '9f0c3f82aedc4e86bbb5f5da41e99090', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 18:56:23.428716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81601) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Sep 07 18:56:23.454264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 18:56:23.455233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application yield resource Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise self.value Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application session.commit() Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 18:56:23.456311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.flush() Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application compat.raise_( Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 18:56:23.457442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 18:56:23.458452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 18:56:23.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [parameters: {'id': '9f0c3f82aedc4e86bbb5f5da41e99090', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:23.462093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 18:56:23.463291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:23.470055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 234/477] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Sep 7 18:56:23 2022] POST /identity/v3/roles => generated 144 bytes in 288 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:23.474918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:23.546070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ee0a0-05be-483d-8742-0e38ef8accae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:23.588209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0222e9-262d-48ea-a968-e5b3cd2edeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.588838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0222e9-262d-48ea-a968-e5b3cd2edeeb None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.589109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0222e9-262d-48ea-a968-e5b3cd2edeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.589373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0222e9-262d-48ea-a968-e5b3cd2edeeb None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.590317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0222e9-262d-48ea-a968-e5b3cd2edeeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:23.591881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0222e9-262d-48ea-a968-e5b3cd2edeeb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:23.602963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79719a-1e91-4ff0-b4f7-7d8c3b3ffee4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:23.648380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.649019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.649290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.649556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.661021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:23.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:23.667616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 18:56:23.668436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:23.670158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 235/478] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:56:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:23.674651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:23.678279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 244/479] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:56:23 2022] POST /identity/v3/users => generated 249 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:56:23.773331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:23.832770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b0cdd-cf31-4a59-8d34-c55d1a4f9cf3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:23.880746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:23.881519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:23.881894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:23.882288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:23.883120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:23.883523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:23.885072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6656191-7917-4913-b359-982a6c564fb1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:23.895626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 236/480] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:25.272542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21b5b1-c4fb-4978-8cd7-d3e75a705223 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:25.273047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21b5b1-c4fb-4978-8cd7-d3e75a705223 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:25.273121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21b5b1-c4fb-4978-8cd7-d3e75a705223 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:25.274681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 245/481] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:25.279959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb5a65-6664-46ac-90de-9cce47308319 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:25.280345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb5a65-6664-46ac-90de-9cce47308319 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:25.280507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb5a65-6664-46ac-90de-9cce47308319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:25.311269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c8eb5a65-6664-46ac-90de-9cce47308319 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:25.312465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c8eb5a65-6664-46ac-90de-9cce47308319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:25.413742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 237/482] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:25.533471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373c942-8668-402d-974e-00e111bd44e3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:25.533651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373c942-8668-402d-974e-00e111bd44e3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:25.533929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373c942-8668-402d-974e-00e111bd44e3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:25.535571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 246/483] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:25.541004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:25.541287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:25.541559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:25.560746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:25.561914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:25.615160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:25.651058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 238/484] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:25.852075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373c942-8668-402d-974e-00e111bd44e3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:25.855223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:25.927136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.012287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373c942-8668-402d-974e-00e111bd44e3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.056515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0763935-7a32-4a88-9809-95c850ed0514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.057152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763935-7a32-4a88-9809-95c850ed0514 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.057426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763935-7a32-4a88-9809-95c850ed0514 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.058156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763935-7a32-4a88-9809-95c850ed0514 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.073420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0763935-7a32-4a88-9809-95c850ed0514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a0763935-7a32-4a88-9809-95c850ed0514 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:26.082238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:26.083164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:26.083987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 247/485] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:26.084737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f1f58-cfce-4cd2-a387-f04e98e69d00 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.085446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd566c7-6626-4982-9989-4fb5a0061505 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.086273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd566c7-6626-4982-9989-4fb5a0061505 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.086948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd566c7-6626-4982-9989-4fb5a0061505 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.089654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 248/486] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:26.090862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd566c7-6626-4982-9989-4fb5a0061505 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.132272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae41e38-0b10-40c6-8d09-36da30323c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.133283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41e38-0b10-40c6-8d09-36da30323c10 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.133940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41e38-0b10-40c6-8d09-36da30323c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.134361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41e38-0b10-40c6-8d09-36da30323c10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.147334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae41e38-0b10-40c6-8d09-36da30323c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-fae41e38-0b10-40c6-8d09-36da30323c10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:26.154174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:26.156096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:26.157972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 239/487] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:26.159576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.163870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.163870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.171843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:26.182856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:26.185511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:26.222109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:26.238712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd566c7-6626-4982-9989-4fb5a0061505 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.280295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 240/488] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:26.281785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.289041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.290931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.290931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.290931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.290931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:26.291278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:26.292536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:26.311906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 249/489] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:26.319642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.479232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168b93e-cae0-4dd6-9360-1a33063900eb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.512252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7094d48-c18f-419a-a84d-15a1e4f22c8e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.517176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.517979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.518399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.518836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.519638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:26.520195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:26.521722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1b07-23bd-4a2c-92f9-eb6ad9fa1288 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:26.531647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 241/490] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:26.533098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20356-e063-481b-9939-326f5ff0b53a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.533528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20356-e063-481b-9939-326f5ff0b53a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.533939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20356-e063-481b-9939-326f5ff0b53a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.535808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 242/491] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:26.537046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20356-e063-481b-9939-326f5ff0b53a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.548197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:26.561987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.562676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.562952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.563367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.576239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:26.577055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:26.578878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f91f4572-aac8-4c63-aed0-ac923f6ff648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:26.594709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:26.595740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:26.596601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:26.596601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:26.596601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:26.596601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 250/492] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:26.597274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.597578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.597832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.616413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:26.617611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:26.660236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20356-e063-481b-9939-326f5ff0b53a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.702574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 251/493] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:26.702574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.706020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.706763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.707163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.707484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.720096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:26.721544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:26.740718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:26.741618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:26.742484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:26.742484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:26.742484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:26.742707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 243/494] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:26.743613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.821654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d05558-5471-43a3-b079-e27e0a767006 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.831536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:26.853660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:26.865076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.876187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.876187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.876187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.877850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56000b-9d9e-459c-b8b7-4726c03a196e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:26.879469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:26.884337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:26.885204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:26.886012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 252/495] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:26.886765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:26.928431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:26.929069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:26.930239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:26.930519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:26.931209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:26.931514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:26.932908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:26.951302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 244/496] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:26.952740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.001768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301933ec-50bb-4f38-92a2-76af4d9206c7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.033995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.034622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.034899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.035170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.035805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:27.036360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.037702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f44a4c-2399-47fa-9e94-e1cce4e66f81 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.049691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 253/497] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:27.050932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea3620-e736-46bb-a58f-9bfe305cdf25 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.051217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea3620-e736-46bb-a58f-9bfe305cdf25 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.051491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea3620-e736-46bb-a58f-9bfe305cdf25 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.052676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 254/498] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:27.053370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea3620-e736-46bb-a58f-9bfe305cdf25 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.062097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca88e-619a-43a6-8c03-b9b5aae7450b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.081129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.095455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.096055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.096327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.096597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.097250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:27.097548 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.098949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.107206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 245/499] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:27.108020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.110495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.157437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea3620-e736-46bb-a58f-9bfe305cdf25 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.192350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.193122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.193690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.193965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.201431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.202891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.208418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacd0b7-0b88-4e1b-9317-7dc35c7ca5dd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:27.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:27.217286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:27.218184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 255/500] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:27.218783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.250662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caee17-6bef-4c51-9337-281110b8811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.251288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caee17-6bef-4c51-9337-281110b8811a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.251563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caee17-6bef-4c51-9337-281110b8811a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.251884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caee17-6bef-4c51-9337-281110b8811a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.261893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5caee17-6bef-4c51-9337-281110b8811a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f5caee17-6bef-4c51-9337-281110b8811a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:27.267686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:27.268590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:27.269402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 246/501] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:27.270054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.270141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.270454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.288566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:27.289807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:27.313545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.329517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2f86e-8789-4325-9f51-3b8ae62562d5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.344742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.354793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 247/502] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:27.355596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.364863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.365834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.366111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.366446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.375847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.377274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:27.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:27.391583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:56:27.392467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:27.392966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 256/503] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:27.393730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.465287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33bfd9-84df-4137-bf92-3cd968248cfb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.497240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c4d46-ef1a-404f-89db-c90165c7a696 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.503232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.503795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.504464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.504739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.516103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.517558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.531895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.532494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.532770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.533045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:27.533204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:27.534100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:27.535513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:27.535513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:27.535513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:27.535715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:27.535715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.535853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 248/504] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:27.535853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.540171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.582102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 257/505] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:27.582990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.588810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.590362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.646213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764050dc-a349-49e0-a745-4bf7c8214342 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.680529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.681122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.681396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.681667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.682323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:27.682649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.684013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.685706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c07758-8b82-43fe-83c1-ba0fe52878d2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.691878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 249/506] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:27.692661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.724620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.725211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.725484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.725749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.726469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:27.726788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.728213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.745689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 258/507] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:27.746476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.794845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477db36-e62d-4822-8e67-868a5dc5adf1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.801327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.805910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:27.828352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.828959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.829235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.829510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.830147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:27.830450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.831867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.844255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd4c5-fb07-4493-8e4f-9a0826aa6213 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.845060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 250/508] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:27.845742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:27.878454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.879087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.879366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.879640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:27.946163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:27.947644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:27.955293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c3f71-400c-4985-b93e-d551193b9cea None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:27.988599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:27.989220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:27.989497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:27.989769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.000347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.001780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.008070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:28.016859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:28.017773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:28.018672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:28.018672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.018672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:28.018672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 251/509] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:28.018890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.042544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.048046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 259/510] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:28.049136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.122415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4159c9-2dc5-4972-9244-1ac0da225272 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.149666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d52ef-fddd-49c7-b120-2479a8f236ed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.155282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.155897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.156172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.156441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.166714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.168144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:28.181821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:28.182762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:28.183633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:28.183633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.183633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:28.183797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.183880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 252/511] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:28.183946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.184019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.184095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.184095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.217673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.257648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.258250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.259170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.290153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd091-1565-473a-9af6-0abd0c0ff2f7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.322480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.323053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.323328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.323596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.331441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.332865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:28.345108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:28.346047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 253/512] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:28.346985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.362028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 260/513] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:28.362906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.423251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.453236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad70c59-86b9-4dcc-a2b0-fe18b444b941 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.459953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9409-d273-4c75-951f-18f5ada2417c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.481530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.486906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.487506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.487784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.488057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.488696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:28.489052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.490446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.495517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.496134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.496404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.496711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.502506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 254/514] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:28.503322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.510919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.512906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:28.527062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:28.528541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:28.529377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:28.529377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.529377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:28.529377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 261/515] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:28.529599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.604424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2b82d-74d0-45af-8433-810e955ca4e0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.631902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f18af0-963b-46f5-917b-a3dda64d725b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.637809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.638406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.638706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.638982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.639618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:28.639921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.641281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.646017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.652194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 255/516] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:28.653601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.672363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.672980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.673257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.673535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.674267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:28.674605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.676051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.694134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 262/517] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:28.694954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.705921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.757053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef205-48cc-4157-8fdf-c959ab653088 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.790432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.791080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.791383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.791657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.794978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621646ce-2a6f-4ab7-8fd5-00e9aa13aef3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.802382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.803860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:28.817490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:28.818439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:28.819301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:28.819301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:28.819301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:28.819301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 256/518] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:28.819523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.831757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.832356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.832629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.832895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.833567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.835001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.908585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 263/519] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:56:28 2022] POST /identity/v3/users => generated 247 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:28.909442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:28.916452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abf2fb-ee5d-44fe-a409-02bb9758c384 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:28.918172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.951288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:28.959256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:28.959878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:28.960158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:28.960432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:28.961275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:28.961849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:28.963009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:28.982877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 257/520] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:28.982877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.045216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c69ab-554d-4234-9d70-5a25eed7ca82 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.097568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.098445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.098893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.099347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.103822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff603-32c5-401e-aec9-c9cbb8366fcb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.107356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.108649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:29.121469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:29.122980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 18:56:29.124021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:29.127100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 264/521] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:29.128142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.146785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.147645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.148077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.148348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.182629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.225359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.228196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.229670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.241076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c7e2-219e-4487-80e6-a9520d92c163 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.277491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.278525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.278840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.279117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.280087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:29.280577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.281961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.296158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 265/522] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:29.296978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.352978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 258/523] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:29.353859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.408186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ce3b0-5912-420b-a957-f579bc5f9410 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.417253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.445919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.446471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.446786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.447059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.454359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.455829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.458644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375773a-7e68-46b7-ac1b-4d1e7ec51d87 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:29.470298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:29.471290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 266/524] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:29.477303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.486830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.503861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.504472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.504745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.505013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.505742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:29.506048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.507578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.544839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 259/525] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:29.546033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.585356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f682f4-f5f7-491d-83e4-1ea174d20520 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.619948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.620568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.620838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.621102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.628188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.629610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:29.641768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:29.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:29.643613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 267/526] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:29.644073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.657743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a24c-dba4-483d-bd2d-f90f6dd1ce36 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.682132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.698158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.698747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.699029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.699306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.709381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.710907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.722947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:29.726387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:29.727297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:29.728141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:29.728141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:29.728141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:29.728141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 260/527] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:29.728365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.759541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092e920-dae8-45b0-b261-113d3d2608a3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.798124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.798779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.799213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.799489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.800294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:29.800604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.802021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.825660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 268/528] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:29.825660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:29.838817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9802d-ae6e-46ad-9022-a1e69f9374c0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.873500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.874108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.874386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.874684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.916968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.934770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5205616-03d8-4449-83b7-809e8aa52914 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:29.943379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.944826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.962508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:29.971996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:29.972741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:29.973023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:29.973296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:29.976443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:29.976443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:29.976443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:29.991507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 269/529] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:29.991507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.054431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 261/530] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:30.054431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.102338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c7892-57c2-4b8c-8b08-38949a53acb9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.120313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:30.139804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.140567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.141165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.141443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.142354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:30.142694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.144059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.153760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61046fdc-e715-4b2c-8c70-3e98221efd3b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.162323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 270/531] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:30.162323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.189342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.189950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.190244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.190513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.191287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=a500e08c87dc4e45899d7408e7b906cc) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:30.191598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.193023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.198600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 262/532] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/role_assignments?user.id=a500e08c87dc4e45899d7408e7b906cc&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:30.199391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.228781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:30.297211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36388119-7f54-4912-90ea-d31a05854ce1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.336760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.337384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.337662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.338076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.338895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=885f09bf588a442e931aae0da51d0ee2) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:30.339205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.341303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.354764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 271/533] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/role_assignments?user.id=885f09bf588a442e931aae0da51d0ee2&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:30.357203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.368283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d83bcf-acc6-4101-ab00-a7844a07d03d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.451706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.452352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.452632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.452907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.461403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:30.474852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.476327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.484337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8202af6-b84b-4b74-aa56-e7a170852670 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:30.508283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:30.509256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:30.510099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:30.510099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:30.510099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:30.510099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 263/534] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:30.515178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.518359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:30.584101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.584745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.585018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.585285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.596401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.597835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:30.610705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:30.611596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:30.612528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 272/535] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:30.619756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.653974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff882b8-a951-41fb-9c55-ec9c64f2f420 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.719117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.719739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.720020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.720294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.720946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:30.721255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.722695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.748969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca876df3-a81d-4317-b733-2b4b7f3213ff None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.756867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 264/536] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:30.762418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.792051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:30.792711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:30.792987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:30.793863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:30.794576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:30.794902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:30.798618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:30.812970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 273/537] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:30.821636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:30.862125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:30.884270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:30.960299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1218-255e-4095-9d12-92cc1f5eb53b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:30.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f8397-9473-4ed1-b33d-4df7b20c9452 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:31.023329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:31.024281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:31.024759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:31.025234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:31.026161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=2f6d9e85ceed45e186421315737f9e56) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:31.028674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:31.030563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:31.041272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 274/538] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/role_assignments?user.id=2f6d9e85ceed45e186421315737f9e56&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:31.053392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:31.054390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:31.054926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:31.055397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:31.067680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:31.069476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-39de55ac-c707-4d83-8cb3-b089002a06b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:31.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:31.094492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:31.095375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 265/539] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:31.100860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:31.234982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:31.240526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1a1d-540f-4426-9b51-9625c31a06c1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:31.257067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28b5b-1a9d-460b-aa55-6ca16d07f3f0 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:31.257380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28b5b-1a9d-460b-aa55-6ca16d07f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:31.257616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28b5b-1a9d-460b-aa55-6ca16d07f3f0 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:31.259294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 266/540] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:31.265722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:31.265996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:31.266278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:31.279246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:31.280254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:31.280752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:31.281232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:31.282579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:31.283052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:31.285074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:31.302679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 275/541] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:31.305734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:31.319006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:31.320214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:31.410639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d5dc-6a9b-4a07-8959-b836355a6aaa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:31.446652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:31.447328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:31.447605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:31.447882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:31.448615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=4267b9d353724a9fab912e6bd0c2ab2c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:31.448925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:31.450383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:31.460599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 276/542] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:31 2022] GET /identity/v3/role_assignments?user.id=4267b9d353724a9fab912e6bd0c2ab2c&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:31.469309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 267/543] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:31.860435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:31.883567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:31.966030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb65fb-6c47-4a31-98e1-ef110b6b62b1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:32.001807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:32.002447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.002754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.003030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.014113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:32.019535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:32.027262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:32.029214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 277/544] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:32.030826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:32.096321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:32.183325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb317ec2-5b37-4dbc-9d17-7fa29bcaf425 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:32.245255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:32.245875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.246152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.247003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.247684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:32.247987 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:32.249356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:32.261707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 268/545] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:32.267467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:32.360138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:32.380171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b81fdd-9552-433e-89e4-c075cc555313 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:32.416016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:32.416647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.416925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.417200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.428179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:32.429626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:32.444269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:32.445232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:32.446056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:32.446056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:32.446056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:32.450516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 278/546] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:32.451888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:32.570866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:32.574203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d9b1-dd69-4aef-9ede-4f4437259613 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:32.617738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:32.618413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.618975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.618975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.619619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:32.619923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:32.621844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:32.638677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 269/547] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:32.644745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:32.769580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749522-0c4c-4141-aef1-f62a71926e9a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:32.800604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:32.812547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:32.812547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.812547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.812547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.823562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:32.825677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2a74067f-f11c-4aef-8fd7-11ff79516153 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:32.839838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:32.841378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:32.842273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:32.844735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 279/548] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:32.850232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:32.927263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8f334-f747-419e-8bdc-1d0edc87ed7f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.928210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8f334-f747-419e-8bdc-1d0edc87ed7f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.928676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8f334-f747-419e-8bdc-1d0edc87ed7f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.930213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 280/549] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:56:32.940677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c2508-21bd-4c6a-8d82-643de8ada038 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:32.941233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c2508-21bd-4c6a-8d82-643de8ada038 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:32.941702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c2508-21bd-4c6a-8d82-643de8ada038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:32.978586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-691c2508-21bd-4c6a-8d82-643de8ada038 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:32.979803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-691c2508-21bd-4c6a-8d82-643de8ada038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:32.998702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0f97c-fbd8-4343-a8a7-00361803fd61 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.038889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:33.039428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.039708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.040200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.041380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:33.041704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:33.043268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:33.070680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 270/550] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Sep 7 18:56:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:33.072785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:33.096317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:33.172958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 281/551] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:33.174270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce8c80-8a8a-4ef1-b934-b5c58df2b9f9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.174577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce8c80-8a8a-4ef1-b934-b5c58df2b9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.174856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce8c80-8a8a-4ef1-b934-b5c58df2b9f9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.176589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 282/552] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:33.181667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.189821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.189821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.191965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592c363-d700-49ee-b7e2-88467cde8eae None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.202707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:33.204273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:33.237512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:33.238207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.238493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.238790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.311125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 283/553] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:33.321272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:33.322827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:33.342045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:33.432401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 271/554] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:33.435672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:33.490866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:33.539683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cabe41-4899-4e58-b15c-525092fb1690 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.576561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:33.577202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.577480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.577757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.579748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:33.591059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:33.593686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:33.631788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:33.632841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:33.634182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:33.634182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:33.634182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:33.634182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 284/555] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:33.640881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:33.765445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7baced-398e-4c66-8bd8-7ef76efcdc06 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.790528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:33.791661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2f4e-a064-454b-b3f9-4e72f8688de7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.795016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:33.814054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:33.814890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.815224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.815768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.830363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:33.830715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:33.831538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.832026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.832567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:33.833456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:33.834005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:33.835317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:33.836273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:33.837078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 272/556] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:33.838284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:33.838884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:33.854339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 285/557] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:33.855793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:33.956561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d793ca-d919-4c50-9afa-2e1fabbfba04 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.971571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b4faa-6b09-4718-b97e-c2a3a6ae0b78 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:33.990419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:33.991069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:33.991341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:33.991607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.001619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:34.006296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:34.007703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:34.008521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:34.009140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.009140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.009777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.009777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.010284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:34.010616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.012947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 273/558] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:34.013076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670e6c5-2554-454b-bf16-1fe665d1d56d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.013535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.025541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 286/559] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:34.025541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d081b95-1ece-488e-a9ae-7ec0037afc5c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.025541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d081b95-1ece-488e-a9ae-7ec0037afc5c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.025541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d081b95-1ece-488e-a9ae-7ec0037afc5c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.025541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 287/560] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:34.026068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d081b95-1ece-488e-a9ae-7ec0037afc5c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.055056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.060065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.119161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242736-7d62-4b36-8b36-33f857e85e9a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.131387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d081b95-1ece-488e-a9ae-7ec0037afc5c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.154182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.154863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.155141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.155416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.162112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.163587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.164344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.164935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.165210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.165472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.166122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:34.166495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.167843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54627bf3-9752-40fa-8c03-4144184e0d2f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a86fc4a4-8824-4ead-be95-fb31fcf6702b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:34.175327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:34.176354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:34.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:34.177714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 288/561] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:34.177798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 274/562] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:34.177867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18eb1d-e11a-4f27-9815-4abda38e8e41 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.177867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18eb1d-e11a-4f27-9815-4abda38e8e41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.177867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18eb1d-e11a-4f27-9815-4abda38e8e41 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.178057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.178057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.178178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.182827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 289/563] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:34.182827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18eb1d-e11a-4f27-9815-4abda38e8e41 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.197682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:34.198900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:34.266254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 275/564] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:34.267123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.280982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.294775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.294996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18eb1d-e11a-4f27-9815-4abda38e8e41 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.330917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.331529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.331805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.332075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.342920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.344348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:34.359121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:34.360084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:34.360988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:34.360988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:34.360988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:34.360988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 290/565] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:34.360988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.375760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14e882-7d24-411b-bb6b-a7d0fdf21deb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.411471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.412080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.412358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.412631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.413317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:34.413621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.415158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3d38-0658-4e0e-b35c-3e0647c3645b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.438082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 276/566] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:34.438082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.438082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.438082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.452416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:34.453590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:34.476436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de293-7bbe-4a0f-ac22-4e2325b41f67 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.513698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.514309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.514610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.514889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.525400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.527549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.528989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.531752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 277/567] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:34.532432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:34.547221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:34.548223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:34.549091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:34.549091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:34.549091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:34.551059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 291/568] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:34.552890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.555900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.646608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6c6f4-a928-4012-a44a-5970ee74a1af None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.657551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad95f01-1a1e-4eb6-9ea5-44ef81b1fa34 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.682368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.683294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.684384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.684839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.685696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:34.686000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.687423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.693800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.694452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.694759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.695037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.695765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=42374efb278647738c5834bdb80db3cb) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:34.696063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.697485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.703375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 292/569] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/role_assignments?user.id=42374efb278647738c5834bdb80db3cb&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:34.704312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.704640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 278/570] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:34.705524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.780530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.819024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:34.831789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b1a54-8536-43a7-bc90-fc89547e39f4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.841537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2601b8-b478-4aee-980a-7c11761eea8e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:34.868560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.869208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.869493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.869768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.878355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:34.879004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:34.879286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:34.879559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:34.880215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:34.880521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:34.881718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:34.881909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:34.887646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:34.888598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 293/571] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:34.891591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.901212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 279/572] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:34.902381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:34.999714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d93a9-127a-4419-ba17-e73ade7eac06 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.001222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac9248-52cd-4e95-8eb1-46f38716b6d4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.017066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.036199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.036830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.037108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.037383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.039126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.039751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.040029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.040614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.045093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.046338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.047836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.051740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:35.056568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.057547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:35.058380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 280/573] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.060202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:35.061182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.062297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 294/574] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:56:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.071650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.167562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa095a0-6a8c-4f6c-8fc8-c5acfbe2dadd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.184352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28a5f0-3421-42fb-ade4-43baa9172d67 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.206832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.207676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.208113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.208387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.212002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:35.212002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.212002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.219813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 281/575] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:35.220353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.227256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.227966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.228315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.228658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.240860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.242373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:35.254621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.255579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 295/576] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.256538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.290263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.295413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.332705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88c91a-a898-42f0-9f35-47365647b34d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.366672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ddf9-f3e9-479c-9e47-6aaf5ab63d07 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.367642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.368250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.368529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.368802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.369471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:35.369779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.371803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.379814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 282/577] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:35.380603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.410477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.411712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.412172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.412651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.413681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:35.414217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.415896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.438984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 296/578] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:35.439821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.492635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd81b99-af2e-4957-b0b3-7b316e8f0b10 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.516327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.528961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.529594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.529869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.530142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.539715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.540910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.542352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.546992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a9c3d-ce00-474e-942f-aa4e7af688d5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:35.557312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.558256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:35.560280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:35.560280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.560280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:35.560280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 283/579] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.560280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.589697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.590334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.590643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.590917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.591729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:35.592032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.593490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.613382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 297/580] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:35.614228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.673710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34495f7f-45fe-45b3-a480-dcd290fbfa3c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.706350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.707027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.707306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.707576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.718193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.719734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.724457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da68b2b-1962-4bbf-836e-3e7798e77a5a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:35.733946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.735001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:35.735874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:35.735874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.735874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:35.735874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 284/581] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.735874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.763459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.764088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.764458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.764632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.768027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.775018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.776569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:35.792564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.794153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:35.795430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:35.795430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.795430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:35.795430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 298/582] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.795430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.799958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:35.850171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7cc6a-0dee-4204-8d35-4ae0b1d6d2b2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.886409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.887551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.887833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.888102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.888735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:35.889035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.890393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.901226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af612787-7697-4296-aa3f-56e327f052f7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:35.901825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 285/583] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:35.903184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:35.940833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:35.941458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:35.941734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:35.942003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:35.953925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:35.955471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:35.970105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:35.971099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:35.972004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:35.972004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:35.972004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:35.972004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 299/584] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:35.972004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.002613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.017407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cbaee-6721-462e-a17b-2e7add2076c5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.040258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.054005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.054638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.054917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.055778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.055890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.056165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.057519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.069309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 286/585] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:36.070102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.093184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276816d7-ad02-45a5-a64a-aebe2fb01ac9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.129245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.129775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.130054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.130329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.131015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.131317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.132659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.150023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 300/586] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:36.150023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.177519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faf210-f9aa-4cec-b07e-e37c7980f584 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.215423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.216270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.216554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.216889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.224547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.225954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:36.237261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:36.238193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:36.239167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 287/587] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:36.239750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.249369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.261241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d896b31-2c84-4a79-baa6-99bceb920db7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.279868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.298742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.299347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.299634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.299915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.300524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.300823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.302165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.314196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 301/588] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:36.315024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.350201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e670158-45b7-4826-8522-08f238446ad3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.386587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.387846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.388127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.388399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.396769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.398237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:36.409970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:36.411234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 288/589] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:36.412247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.436161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2c69f-24f4-411e-a533-1e6168e9c875 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.469780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.470408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.470715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.470986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.477945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.479336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.481526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:36.490720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:36.497150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 302/590] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:36.498173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.524803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008308c-dcbd-41f2-9aaf-9bc5977cac81 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.530669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.560710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.561324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.561592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.561867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.562615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.562924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.564336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.581639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 289/591] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:36.582728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.606594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c954a7-0236-4e62-86fa-e3174382ca7a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.644132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.644760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.645040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.645389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.651970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.654025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:36.665455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:36.666336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:36.667530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:36.670965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 303/592] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:36.671535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.694303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd21938-85c9-43e8-a807-448a0e616e0d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.720036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.732240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.732240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.732512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.732748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.733455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.733760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.735225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.754853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 290/593] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:36.759724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.783822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.792883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77536-f986-45b8-8279-2a27f1b1c360 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.832895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.833505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.833780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.834052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.834768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.835074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.836487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.850362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 304/594] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:36.851211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.883385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9114e-7025-41a8-88e3-76508b8fcf3e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.917819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.918436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.918757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.919026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.928609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.930041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:36.944231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:36.945304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:36.946348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:36.946348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:36.946348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:36.946348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 291/595] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:36.946654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:36.957785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d283-4f2f-49e7-8545-a43029ee7a0e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:36.980707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:36.992480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:36.992989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:36.993265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:36.993538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:36.994226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:36.994531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:36.996023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.008236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 305/596] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:37.009375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.029749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.045758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550026-dea0-4486-8145-70ab10f6223b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.080331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.080950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.081229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.081503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.092569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.094021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-113bd784-1593-42a4-ada2-510c92abc1fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:37.107685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:37.108797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:37.109963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:37.109963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:37.109963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:37.109963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 292/597] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:37.110190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690a61-3d58-4d23-b6f9-006a8b7bded2 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.110424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690a61-3d58-4d23-b6f9-006a8b7bded2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.110720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690a61-3d58-4d23-b6f9-006a8b7bded2 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.111925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 293/598] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:37.112599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690a61-3d58-4d23-b6f9-006a8b7bded2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.115908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832f1b-4664-4a3b-bb72-914cf67f445f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.149790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.150433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.150744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.151019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/a500e08c87dc4e45899d7408e7b906cc/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.164984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=a500e08c87dc4e45899d7408e7b906cc, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.166391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.196693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.212156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690a61-3d58-4d23-b6f9-006a8b7bded2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.227521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 306/599] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:37 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/a500e08c87dc4e45899d7408e7b906cc/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:37.228325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.255090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.256010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.256296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.256574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.257254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:37.257563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.259033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.277354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 294/600] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:37.277354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.306082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.347506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661f999-a464-49fc-8288-6c94bd04093a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.388771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733f54-c523-40f8-a253-8626780f6ea7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.394418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.395243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.395614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.395986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/885f09bf588a442e931aae0da51d0ee2/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.416834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=885f09bf588a442e931aae0da51d0ee2, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.420097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.428567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.435787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.435787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.435787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.435787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:37.435787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.435787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed4a6-a7c1-402b-8cd4-ec7c500ae83d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.444198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 295/601] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:37.445364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.445688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.445963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.457833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.463657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:37.464983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:37.488527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 307/602] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:37 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/885f09bf588a442e931aae0da51d0ee2/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:37.489899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.550363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 296/603] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:37.551331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.606013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd408530-1f03-4d4d-8d22-23388168ce77 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.625393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.645934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.646990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.647474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.647942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.659027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.661062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:37.676787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:37.677767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:37.678716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 308/604] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:37.682029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8686c-2933-41eb-b135-cf144797f6da None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.684061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.727752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.728492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.728774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.729043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.738382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.744954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.747361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:37.764240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:37.765222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:37.766113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 297/605] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:37.771622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.835952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217b487-f26b-4f28-9609-8887ba8afade None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.872335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601363a-5763-4493-908e-ca3860102472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.872937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601363a-5763-4493-908e-ca3860102472 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.873208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601363a-5763-4493-908e-ca3860102472 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.873475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601363a-5763-4493-908e-ca3860102472 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.874162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601363a-5763-4493-908e-ca3860102472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:37.874461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601363a-5763-4493-908e-ca3860102472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.876098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601363a-5763-4493-908e-ca3860102472 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.889121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 309/606] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:37.894448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601363a-5763-4493-908e-ca3860102472 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:37.914868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232288-d617-4f15-9dd9-8236845ff0d9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:37.941142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:37.976657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:37.977273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:37.977547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:37.977818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:37.978515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:37.978850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:37.981384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:37.998034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 298/607] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:37.999473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:38.008985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601363a-5763-4493-908e-ca3860102472 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:38.053539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:38.054222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:38.054483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:38.054798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/2f6d9e85ceed45e186421315737f9e56/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:38.073540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=2f6d9e85ceed45e186421315737f9e56, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:38.073779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:38.075678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:38.121211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cae90-8aca-44cf-a210-245748f6a31a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:38.163121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 310/608] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:37 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/2f6d9e85ceed45e186421315737f9e56/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:38.165643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:38.169975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:38.170783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:38.171069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:38.171577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:38.194724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:38.200886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:38.201876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:38.207535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 299/609] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:38.208743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:38.342415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:38.367767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a3ddd-8201-4a58-9d0d-fdd494d8dbdf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:38.416072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7dee-5deb-43d7-b67b-79d1a0c96829 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:38.433121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:38.433828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:38.434107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:38.434380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/4267b9d353724a9fab912e6bd0c2ab2c/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:38.445298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:38.459056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=4267b9d353724a9fab912e6bd0c2ab2c, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:38.461256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:38.475132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:38.475132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:38.475132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:38.475132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:38.475466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:38.475466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:38.475466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:38.481328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 300/610] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:38.486882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:38.531044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 311/611] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:38 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/4267b9d353724a9fab912e6bd0c2ab2c/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:38.629191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eee116-5a2e-42f2-b0cc-6189e4f72546 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:38.667493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:38.669199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:38.669682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:38.670158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:38.682607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:38.684632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:38.700333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:38.701347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:38.702239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:38.702239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:38.702239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:38.704502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 301/612] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:38.710123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:38.838677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87aa4-9482-40fe-b8f9-e876cb375eb0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:38.879524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:38.880133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:38.880404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:38.880671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:38.881314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:38.881607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:38.882991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:38.885418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:38.896121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 312/613] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:38.901482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:38.964750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:39.061888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37858b3-8775-465c-b8de-6e2645c1ce38 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:39.099617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:39.100253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:39.100530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:39.100809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:39.109267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:39.110755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:39.122805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:39.123973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:39.124993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 302/614] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:39.129668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:39.278128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc16a4-64db-4e41-adcc-f369319c48f1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:39.322510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:39.323151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:39.323420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:39.323683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:39.324449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:39.324744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:39.326147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:39.353491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 313/615] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Sep 7 18:56:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:39.355882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:39.481429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:39.494885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ee5d-e62f-4df8-91c6-21c96d61220b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:39.545554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:39.546239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:39.546521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:39.546816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:39.558142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:39.559661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-59a8c3e1-c8d1-41ab-b5d0-bfba7428bbda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:39.579951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:39.580966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:39.581842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:39.581842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:39.581842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:39.586593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 303/616] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:39.590332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:39.634040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcbf66-8a18-4cda-ac5b-b3b0b96704ea None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:39.634350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcbf66-8a18-4cda-ac5b-b3b0b96704ea None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:39.634645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcbf66-8a18-4cda-ac5b-b3b0b96704ea None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:39.636074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 304/617] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:56:39.641795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:39.642388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:39.642899 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:39.670544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:39.672169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:39.701485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:39.786543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f4fd0-4ba7-4167-83d5-ec08e80e6209 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:39.809278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 305/618] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:39.834641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:39.835703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:39.835703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:39.837571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:39.838238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:39.838562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:39.839977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:39.857967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:39.868825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 314/619] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:39.959570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:39.998283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316d698-507e-4fa6-b544-e402a921f3c6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:40.050049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:40.050667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.050957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.051228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.059871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:40.064801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:40.077230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:40.078151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:40.079079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:40.081123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 306/620] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:40.087000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:40.119891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:40.240821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9c7f-992e-4b58-8ef7-6b25fb849d49 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:40.264886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:40.307227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:40.309168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.309652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.310295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.312365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:40.312885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:40.314684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:40.327992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bee959-9bb0-4c26-b104-c6ae81598ca6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:40.344491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 315/621] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:40.350534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:40.380633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:40.389920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba035a1-dafd-403a-844d-6344adeae78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:40.392058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba035a1-dafd-403a-844d-6344adeae78d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.392938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba035a1-dafd-403a-844d-6344adeae78d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.393442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba035a1-dafd-403a-844d-6344adeae78d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.408816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba035a1-dafd-403a-844d-6344adeae78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0ba035a1-dafd-403a-844d-6344adeae78d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:40.416814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:40.418274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:40.419182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:40.421673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 307/622] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:40.425140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2215be-7988-4412-a3d8-f349a07be4f2 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.425606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2215be-7988-4412-a3d8-f349a07be4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.426113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2215be-7988-4412-a3d8-f349a07be4f2 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.428699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 308/623] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:40.429650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2215be-7988-4412-a3d8-f349a07be4f2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:40.497747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7935cc-db8f-4188-8d3b-48b1d4634bb9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:40.538153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:40.539084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.539508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.539949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/42374efb278647738c5834bdb80db3cb/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.555930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2215be-7988-4412-a3d8-f349a07be4f2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:40.557362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=42374efb278647738c5834bdb80db3cb, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:40.559068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:40.592703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:40.593316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.593589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.593857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.594511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:40.594842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:40.596217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f95840-489d-4fdd-953e-86aebe2c48aa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:40.604695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 309/624] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:40.605675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.605953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.606221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.626358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:40.627565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:40.646359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 316/625] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:40 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/42374efb278647738c5834bdb80db3cb/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 18:56:40.647692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:40.701091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:40.729697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:40.775556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 310/626] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:40.827429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ff07d-5e78-410b-967b-91d08f348fc5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:40.867597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:40.868258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.868539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.868810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.881650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:40.883158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-983406e9-99bb-4bcb-b18a-a73f9fb65212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:40.900195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:40.901214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:40.902669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:40.902669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:40.902669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:40.902669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 317/627] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:40.907871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:40.977288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfe836-067d-42aa-a8f7-56897b59572c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.977573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfe836-067d-42aa-a8f7-56897b59572c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.977848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfe836-067d-42aa-a8f7-56897b59572c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:40.980656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 318/628] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:40.987980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:40.989220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:40.989602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.017851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:41.019445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:41.029196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d616d9-4573-4097-a5c8-eaeff384ea47 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.074532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.075163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.075440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.075709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.076368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:41.076808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:41.078798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:41.091398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 311/629] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:41.097465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.124765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 319/630] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:41.152320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.156149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.170951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.233017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bacc6-7333-4ef9-9344-4e8d9c3e1d45 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.270945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.271573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.272009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.272362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.281191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:41.282884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3366537d-6a1b-415a-9cd3-c7280397a699 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:41.299447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:41.300428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:41.301365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 312/631] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:41.307903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433e809-c752-430a-9320-792b9230c9c4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.308755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433e809-c752-430a-9320-792b9230c9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.309036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433e809-c752-430a-9320-792b9230c9c4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.310673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 313/632] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:41.311386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433e809-c752-430a-9320-792b9230c9c4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.348935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe4550-42ef-4873-9e34-33c196d7882c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.394643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105648-6845-49fd-a522-6fa971920193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.395909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105648-6845-49fd-a522-6fa971920193 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.396974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105648-6845-49fd-a522-6fa971920193 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.397450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105648-6845-49fd-a522-6fa971920193 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.412033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105648-6845-49fd-a522-6fa971920193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ef105648-6845-49fd-a522-6fa971920193 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:41.419363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:41.420295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:41.421137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:41.423123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 320/633] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:41.424200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.424483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.424862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.438085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433e809-c752-430a-9320-792b9230c9c4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.440512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.447304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:41.448520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:41.477387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.478001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.478347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.478603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.480812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:41.481126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:41.482577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:41.503222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 314/634] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:41.503876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.519798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.527927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 321/635] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:41.533711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.633180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea3abc-5b92-45ad-bbb0-8b4867147347 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.653850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e9307-b674-4569-a17f-f96717dda899 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.672504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.673216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.673587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.673892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.684644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:41.689592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:41.690525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:41.691560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:41.693382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 315/636] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:41.694445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.698265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.699238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.699738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.700218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.701103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:41.701659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:41.703246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:41.718650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 322/637] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:41.719905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.723324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.792247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.805212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b23d52-ec8a-4d43-9526-a8e2ec693719 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.841425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481077b4-abc9-4472-8bd4-ee1a207578c7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:41.842384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.843184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.843464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.844513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.876244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:41.876857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:41.877131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:41.877399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:41.878060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:41.878359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:41.879753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:41.893357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 323/638] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:41.893357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:41.929525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:41.931214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:41.955911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:41.997391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43620a-2f6c-45a5-98e3-7b3139a4a300 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.019348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.037417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.038405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.038742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.038989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.039307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 316/639] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:42.040162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.057862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.060179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:42.078750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.079866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:42.080784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:42.080784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.080784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:42.082593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 324/640] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.084207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.144864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4065be-5c3b-40b4-9257-3f3a25425501 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.167774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.180387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.181017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.181288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.181560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.192160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:42.196779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.197783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 317/641] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.199653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.205407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59730751-6a5c-42a9-908b-6001b8331cd8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.246330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.246985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.247540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.247540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.261509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.263839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:42.281914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.282969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:42.283919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:42.283919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.283919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:42.287914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 325/642] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.288538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.311851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb944533-e49d-412f-886c-538c1bab7f86 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.315646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.349538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.350165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.350441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.350736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.361459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.362962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:42.377711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.378972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:42.379951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:42.379951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.379951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:42.383339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 318/643] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.384116 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.393852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29de9-e3a3-47a1-b64d-4d4d310b7a91 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.419266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.431971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.432950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.433204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.433483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.434130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:42.434493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.435977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.448109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 326/644] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:42.449067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.495085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77376990-79ab-4a4c-b9a5-08d33f8fd3a4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.529056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.529659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.529942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.530216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.530909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:42.531215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.532584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.541174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 319/645] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:42.541992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.551517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.568601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb64f29-481c-41b2-9baf-00e21e091252 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.604980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.606169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.606442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.606756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.607394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:42.607693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.609038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.625042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 327/646] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:42.626580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.644599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.650322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f2cc1-ab09-4e62-b708-734fcc48b699 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.687068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.687996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.687996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.688208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.688748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:42.689049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.690406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.702793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 320/647] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:42.703599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.744313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139619f4-01af-48a9-8b51-50871fe06955 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.782421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.783128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.783406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.783670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.792084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.794040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:42.808725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.809918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:42.810926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 328/648] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.811522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.815942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.818745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c00815-14c6-4336-bd32-3e652480ac64 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.859826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.860462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.860787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.861053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.873313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.874799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:42.889487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.890519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:42.891458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:42.891458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:42.891458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:42.895100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 321/649] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.896402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:42.911882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:42.919289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb16de-4a3a-402a-a18a-f9b53e8cd1a3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:42.956311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:42.956863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:42.957182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:42.957457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:42.965293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:42.966857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:42.978168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:42.979194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 329/650] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:42.985257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.012071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6068-8107-4f0e-8834-cc3b6732c08f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.050834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.051895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.052560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.052835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.053114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.061468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.063274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:43.078924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:43.079886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:43.080782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 322/651] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:43.081341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.105853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181070-c32b-48ff-a853-d3d4b19b864b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.142379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.143006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.143287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.143561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.144297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:43.144602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.146020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.160614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.164617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 330/652] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Sep 7 18:56:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:43.165629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.200433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e4fc3-8115-40c6-9ad4-bdd75d6fcd83 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.233368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.233891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.234162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.234430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.235091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:43.235393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.236747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.248758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 323/653] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:43.249543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.271556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29c454-ee22-4b6b-b81d-d6273289273f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.293692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.305425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.306342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.306800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.307075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.307804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:43.308653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.310098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d8df9-1636-4dcb-961f-a027c590dac7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.328384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 331/654] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:43.329576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4738c4-fc9b-488b-a188-1e727fe18cf2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.329860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4738c4-fc9b-488b-a188-1e727fe18cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.330136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4738c4-fc9b-488b-a188-1e727fe18cf2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.334276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 332/655] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:43.334276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4738c4-fc9b-488b-a188-1e727fe18cf2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.350047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb087a2b-57d6-4997-b124-2545e5a5b7b9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.378297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.387607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.388313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.388635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.388986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.389686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:43.390073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.391802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.405072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 324/656] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:43.405800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.440749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4738c4-fc9b-488b-a188-1e727fe18cf2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.475405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055852d-ae17-4846-bd9c-95a499e8e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.476017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055852d-ae17-4846-bd9c-95a499e8e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.476296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055852d-ae17-4846-bd9c-95a499e8e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.476566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055852d-ae17-4846-bd9c-95a499e8e2e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.509147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467e7e7-4c3e-47b0-9f19-bf70fa9d770f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.526146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.542818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.543438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.543712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.543983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.546390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055852d-ae17-4846-bd9c-95a499e8e2e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.548308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055852d-ae17-4846-bd9c-95a499e8e2e4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.551746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.553171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:43.564100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:43.565291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 325/657] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:43.566325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.614922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.650862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 333/658] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:43.652797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.653078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.653348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.671112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:43.671311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca0bfa-fdab-43ab-a654-21d6f0215dd1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.672279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:43.704613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.705211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.705489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.705762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.737128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 334/659] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:43.737941 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.749421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.786307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.787719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.833810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:43.843755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf88f1-5d31-40bd-ae33-8418da88dd19 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:43.882320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:43.882955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:43.883235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:43.883516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:43.884240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=a500e08c87dc4e45899d7408e7b906cc) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:43.884588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:43.886021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:43.892545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 326/660] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:43.892545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.902587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 335/661] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/role_assignments?user.id=a500e08c87dc4e45899d7408e7b906cc&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:43.902587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:43.998860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.029209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5a627-8277-4ea5-9bd1-9bcbc8319030 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.029476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a829-6363-4e3c-a823-52a86aeec945 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.070694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.071240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.071337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.071431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.071513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.071591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.071916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.072008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.072494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:44.072803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.075305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.084465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.086786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.094256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 327/662] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:44.094652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:44.105942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:44.107025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:44.107909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:44.107909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.107909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:44.108515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 336/663] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:44.108612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.108972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.221698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efa48-cb05-4083-b52c-442c1bbf1e2e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.228180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e77e3-d4e9-488e-9867-53c126fd061e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.255458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.261345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.261978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.266202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.266202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.266394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.266394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.266394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.266394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.273069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.274513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.276023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:44.280783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:44.281739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 337/664] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:44.283042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:44.293739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:44.294741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:44.295644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:44.295644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.295644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:44.295644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 328/665] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:44.295884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.358999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.396101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedc4a5-7422-4707-95bc-205f3126de72 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.406113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725850-18fc-4640-814a-09a4efe2420c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.432192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.432818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.433092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.433366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.441151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.441751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.442023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.442298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.442968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:44.443268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.444648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.460965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 329/666] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:44.462271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.501161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.514684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.516123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.578475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2042ed-4f5c-42d0-b1d6-121ae5e52aa6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.596495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.614471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.614471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.614731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.615002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.615761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:44.616070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.617507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.624614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 338/667] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:44.624614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.626511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 330/668] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:44.627431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.715089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.731053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56da2e6-3d4d-45e8-9f91-acaae334f385 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.733100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79956d-3fe4-41db-9786-914d079189e1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.768159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.768401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.768768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.768878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.769062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.769168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.769332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.769429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.769975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:44.770242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.771642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.776094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.778354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.789965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 339/669] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:44.789965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:44.790223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:44.791095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 331/670] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:44.791978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.836543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.902391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109b6c-9cd2-4124-ae92-a6363e09048a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.904639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761cd6a-9111-4b6b-aafb-89a20edc6821 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:44.940228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.940625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:44.941006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.941352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.941446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:44.941741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:44.941842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.942123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:44.952748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.953803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:44.954315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.955386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:44.961264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:44.969543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:44.970880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:44.971991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:44.971991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.971991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:44.972211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:44.978138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:44.979296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:44.979296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:44.979296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:44.979501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 332/671] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:44.979501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:44.980376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 340/672] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:44.980376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.075674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.083822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c43e3-1aed-429a-b4cd-ddd8cc17e006 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.084057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170689-0c78-4b9d-b565-6de2771a1695 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.117934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.118579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.118849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.119166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.120279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.121449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.121730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.122005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.122741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.123039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.124467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.127521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.128947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.142178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 341/673] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.142178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:45.142419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:45.143713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 333/674] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:45.144610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.207302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.255878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcba8e4-3cc9-4cda-bede-9e2c36a460a4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.265324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72a1ad-9c3a-4e36-943c-f7ae1374d0d3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.295471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.296338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.296788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.297060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.297708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.298010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.299412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.301514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.302192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.302621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.302965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.303674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.304051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.305469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.311780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 342/675] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.316786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.321697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 334/676] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.321697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.334405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.423280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a296e-14b5-4646-b3d3-dc6f6e2cf3a8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.426661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62f26e-fb39-4a9f-a5f4-17d46320e307 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.444826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.456994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.457645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.457874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.458151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.458885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.459199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.460621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.464355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.465070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.465458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.466044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.466943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=885f09bf588a442e931aae0da51d0ee2) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.467414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.469482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.472250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 335/677] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.473115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.480757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 343/678] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/role_assignments?user.id=885f09bf588a442e931aae0da51d0ee2&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.481761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.566803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.612572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536871-4fdf-4271-86d3-bf75a760e19f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.624490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048c82-2f62-47a7-b47b-7a4cda0a693d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.652380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.652992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.653372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.653752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.663004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.664676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.676570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.677204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.677482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.677761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:45.679413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:45.680528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:45.681685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:45.684481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 336/679] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:45.685010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.685695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.691670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:45.705213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:45.706326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:45.707479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:45.712646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 344/680] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:45.712646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.773314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.820138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2e0b4-7763-4c97-aa28-19bf5e06a401 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.850119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9ee36-75b3-43a8-a428-26ed1ab73da4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:45.859041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:45.866606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.867450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.868114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.868946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.873596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=2f6d9e85ceed45e186421315737f9e56) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.873596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.873596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.881629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 337/681] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/role_assignments?user.id=2f6d9e85ceed45e186421315737f9e56&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.882974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:45.888086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:45.889560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:45.890055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:45.890588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:45.891648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:45.892200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:45.893895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:45.915205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 345/682] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:45.921655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.011204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7710-86c6-4960-99aa-710f5cadc7e5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:46.056862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:46.057503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.058383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.058691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.059428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:46.059739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:46.061532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:46.080267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 338/683] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:46.086443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.149936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:46.154151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0309-2b9d-4d31-b8b5-a18c802d91ab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:46.171504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:46.202788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:46.203836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.204328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.204814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.233833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff021997-06c0-4a90-9aa5-a40e536dc8d5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:46.274527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:46.275210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.275468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.275732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.276516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=4267b9d353724a9fab912e6bd0c2ab2c) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:46.276878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:46.278591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d6fd7-78c4-4ad0-a4f8-0ccc6757c989 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:46.286445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 339/684] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:46 2022] GET /identity/v3/role_assignments?user.id=4267b9d353724a9fab912e6bd0c2ab2c&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:46.288444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58592002-bb5d-4042-b3e8-1d30024382a3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.289172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58592002-bb5d-4042-b3e8-1d30024382a3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.289605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58592002-bb5d-4042-b3e8-1d30024382a3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.289962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:46.291360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 340/685] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:46.292058 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:46.298485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.298893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.299242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.329436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:46.331771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:46.423140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:46.424615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 341/686] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:46.438608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 346/687] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:46.440005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.520096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:46.580874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbe08b-7817-4e31-9a01-833486e1127f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:46.617005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:46.617990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.618993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.631018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:46.632758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:46.648040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:46.649009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:46.649898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:46.649898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:46.649898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:46.652078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 342/688] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:46.657267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.739580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.775577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e4e07-9fce-4520-8724-e8d313828116 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:46.826320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:46.827060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.827338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.827607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.828261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:46.828737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:46.832512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:46.858996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 347/689] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:46.864603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.903207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572efc-5dc1-43ef-98af-65014c71331c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:46.905127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:46.939922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:46.940941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:46.941426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:46.941912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:46.952298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:46.954941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:46.960672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:46.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:46.962490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:46.965824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 343/690] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:46.971305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:46.997188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58e8d6-0355-45cc-ab17-4a1293ff53c5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.034010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.034670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.034945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.035465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.043186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:47.044787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:47.056665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:47.057599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:47.059111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:47.062748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 348/691] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:47.064231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:47.173491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc7513-7812-4d84-8ea8-06d382eb1b4e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.194122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:47.208740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.212053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.212053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.212053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.212053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:47.212053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:47.215038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:47.223905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7bc62-99f7-429c-b799-bb281bddd244 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.235500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 349/692] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:47.235500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:47.318499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.319217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.319499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.319766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.320446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:47.320748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:47.322167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:47.339799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd711c3-5da8-4c08-82a6-fbf4ce8a354e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.352780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 344/693] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:47.357992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:47.374317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.374953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.375233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.375504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.376231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=42374efb278647738c5834bdb80db3cb) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:47.376539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:47.377983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:47.390697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 350/694] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:47 2022] GET /identity/v3/role_assignments?user.id=42374efb278647738c5834bdb80db3cb&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:47.427017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:47.515921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac4bb-af2a-4f81-9dab-6ba98857b511 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.572046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.572689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.572959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.573303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.588084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:47.590109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:47.608608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:47.609637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:47.610533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:47.610533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:47.610533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:47.610727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 345/695] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:47.615302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:47.686236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:47.764962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4a194-391e-4e3b-92ec-a0bbe5c773dd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.808273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.809126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.809626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.810043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.810897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:47.811335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:47.812827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:47.826272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 351/696] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:47.831376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:47.922376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:47.948470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8c0aa-860a-48d4-9c96-0357d8a50149 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:47.989165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:47.989807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:47.990087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:47.990354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:47.999353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:48.000803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b765d7db-41fa-4b04-9ed7-c6bdb62ddd36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:48.013011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:48.013936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:48.015292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 346/697] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:56:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:48.020064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:48.086886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e380bc-a61e-4095-a74c-868c2d1068e0 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.087210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e380bc-a61e-4095-a74c-868c2d1068e0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.087437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e380bc-a61e-4095-a74c-868c2d1068e0 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.089744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 347/698] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:48.095236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.095520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.095793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.121504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:48.122733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:48.133602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436f778-598b-4d77-9275-c08567d83367 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:48.137177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:48.201060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:48.203049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.203540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.207119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.208190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:48.208764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:48.212363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:48.242602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 348/699] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:48.244382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 352/700] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:56:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:48.250455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:48.329514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:48.371858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faadad-35a7-495e-adbc-860c9937c13c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:48.414115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:48.414800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.415081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.415355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.485233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:48.486736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:48.553691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:48.606518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 349/701] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 18:56:48.611864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:48.628912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:48.795070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c3de9-eaaf-4c43-b50c-41d5318a2ba7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:48.799061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f551b-6dbb-4b2b-beb4-54f5275fe668 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:48.840413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17945252-329c-4df9-adfa-c93065211971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:48.841044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17945252-329c-4df9-adfa-c93065211971 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.841319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17945252-329c-4df9-adfa-c93065211971 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.841587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17945252-329c-4df9-adfa-c93065211971 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.842166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17945252-329c-4df9-adfa-c93065211971 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:48.842870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:48.843500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.843625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17945252-329c-4df9-adfa-c93065211971 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:48.843822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.844236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-17945252-329c-4df9-adfa-c93065211971 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:48.854976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:48.856118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:56:48.856984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:48.857740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:48.857815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 350/702] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Sep 7 18:56:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:48.857890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:48.857960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fda253-357a-480c-840d-a5b39a732212 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.857960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fda253-357a-480c-840d-a5b39a732212 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.858095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fda253-357a-480c-840d-a5b39a732212 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.862738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 351/703] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:48.864721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fda253-357a-480c-840d-a5b39a732212 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1f5a8980-8dfc-4602-b04a-9e91ccf6ae31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:48.875345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:48.877243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:48.878336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:48.878336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:48.878336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:48.878336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 353/704] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:48.878336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:48.882678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:48.882995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:48.904945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:48.909303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:48.911025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:48.923966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:48.987427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fda253-357a-480c-840d-a5b39a732212 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:48.998710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 354/705] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:48.999699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.052334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.052860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.053133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.053403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.054072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:49.054373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.055768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.066018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 352/706] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Sep 7 18:56:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:49.069992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.186884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9cb9-d845-485a-ad41-c601224eff29 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.232059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.232880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.233158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.233426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.234085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:49.234384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.235833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a010a8-7649-4536-8b87-3a28203ac1d8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.248986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 355/707] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:49.250524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672f7c9-aaea-4294-a46e-a9df642a6c6d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.250690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672f7c9-aaea-4294-a46e-a9df642a6c6d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.250893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672f7c9-aaea-4294-a46e-a9df642a6c6d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.256408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 356/708] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:49.257151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672f7c9-aaea-4294-a46e-a9df642a6c6d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.257263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:49.261473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:49.265920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fd9e4-be45-4bf8-85d2-79ae79884d09 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.301301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.302200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.303127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.303590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.304732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:49.305491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.307256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695d301-c6af-41e6-af78-282df695db37 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.316021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 353/709] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:49.322807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.322807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.322807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.347919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:49.350311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:49.386964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672f7c9-aaea-4294-a46e-a9df642a6c6d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.427700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.428542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.428911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.429296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.439196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.441053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.446645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 354/710] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:49.447585 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:49.465335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:49.466307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:49.467232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:49.468944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 357/711] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:49.475637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.575526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:49.607162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:49.634725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330517f3-e9ab-4a69-aeb5-e15c683c7904 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.639554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9030f-ab6a-4011-8008-7a967e1d2d05 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.709262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.709904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.710191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.710458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.712459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.713138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.713418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.713717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.714421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:49.714745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.716177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.734276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.735789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.737951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 358/712] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:49.739166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:49.757431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:49.758490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:49.759578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:49.759578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:49.759578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:49.759578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 355/713] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:49.759822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.855669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848230ec-e21a-47a5-890f-e9ff2cfd48b8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.883175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:49.889521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5b823-f1d3-487b-ade0-7e09de05ae74 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:49.894910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.895559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.895896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.896162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] PATH_INFO: `/v3/services/network` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.896751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.898194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:49.917369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:49.918652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:49.919495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 359/714] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:49.920734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.929211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:49.929814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:49.930092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:49.930370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:49.931136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=a500e08c87dc4e45899d7408e7b906cc) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:49.931445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:49.932924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:49.938924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 356/715] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/role_assignments?user.id=a500e08c87dc4e45899d7408e7b906cc&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:49.939729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:49.955245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:50.043477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc53eaa-a2dd-4839-9891-a15436947a12 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:50.084805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:50.085546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:50.085895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:50.086231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:50.087015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:50.087383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:50.089068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:50.104313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 360/716] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:50.110840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:50.152220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52551dd7-fbbb-45bd-a41e-e10512230a64 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:50.212693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:50.217681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:50.218569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:50.219054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:50.219527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:50.220358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] RBAC: Adding query filter params (name=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:50.220887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:50.222482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:50.227835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 357/717] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:56:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:50.235644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:50.276930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e68c93-8a49-48e0-977f-4db8f4a72223 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:50.328703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:50.329731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:50.330215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:50.330846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:50.332017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:50.333827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:50.343225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:50.379796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce15de-bfb5-4cb0-b77e-a6c034071e97 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:50.416378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:50.417008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:50.417289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:50.417563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:50.418189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:50.418492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:50.420015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587083e6-25c9-4fc2-ac4f-68a353f794a3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:50.424422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 358/718] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:56:50 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:50.425555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaef405-c945-45da-bf9a-b52cf7b75209 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:50.425837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaef405-c945-45da-bf9a-b52cf7b75209 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:50.426117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaef405-c945-45da-bf9a-b52cf7b75209 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:50.427330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 359/719] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:50.433166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:50.433516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:50.433798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:50.454616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:50.455876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:50.470338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 361/720] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:56:50 2022] POST /identity/v3/users => generated 254 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:50.551149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 360/721] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:50.837597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:50.968011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b20a15-7a9b-4688-8242-082db3460061 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:50.975585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:51.015668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:51.016728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:51.017236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:51.017706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:51.039422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:51.044829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:51.045792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:51.046664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:51.050105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 362/722] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:51.055198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:51.123622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:51.181282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06354-bdbf-4b57-95db-da6cbe242efb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:51.222509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:51.223144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:51.223420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:51.223686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:51.224348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:51.224646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:51.226016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:51.238677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 361/723] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:51.241147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:51.368364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef296-7107-4df2-8f4a-a4d8e45957f6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:51.410401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:51.411423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:51.411923 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:51.412396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:51.424131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:51.425871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:51.445788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:51.451191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:51.452094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:51.452094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:51.452094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:51.454087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 363/724] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:51.462143 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:51.606362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b740-d1c1-4628-95ee-5e7ba4820d3c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:51.642665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:51.657096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:51.657713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:51.657991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:51.658263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:51.658946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:51.659254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:51.669891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:51.678216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 362/725] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:51.684128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:51.708087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:51.805034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90558ce-487a-45db-823d-05612c3df882 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:51.841782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:51.842415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:51.842721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:51.842992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:51.850450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:51.851918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:51.864474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:51.865506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:51.866479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 364/726] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:51.872201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:52.001189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3894-c0e0-4366-89fd-fa0f2f85bbb9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:52.047543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:52.048150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.048428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.048699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.049437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:52.049741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:52.051258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23456d4-3954-459f-b43e-cc187bd2324d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:52.082241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 363/727] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Sep 7 18:56:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:52.087835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:52.121807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fee72-3533-425f-bef2-eecfec5b5a57 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.121807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fee72-3533-425f-bef2-eecfec5b5a57 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.121807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fee72-3533-425f-bef2-eecfec5b5a57 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.122665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 364/728] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:52.140555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.140844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.141121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.175564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:52.176768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:52.196993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:52.225036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f51890-c774-4620-941e-4b60dab816e3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:52.273271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:52.304707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 365/729] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:52.314176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:52.314823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.315107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.315383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.421421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:52.422891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:52.498838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:52.535373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 365/730] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 18:56:52.541148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:52.668878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:52.685766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fbc4d-da94-4d09-8bd3-e50fe0141b04 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:52.750830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:52.751914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.752419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.752891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.771147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:52.773071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:52.804480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6abd4f-aa45-49a6-8c65-7ea491b0e281 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:52.808684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:52.809606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:52.810520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:52.810520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:52.810520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:52.819215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 366/731] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:52.824297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:52.847935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9476c-694c-40d9-8492-567cc8d2e2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:52.848557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9476c-694c-40d9-8492-567cc8d2e2b0 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.848840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9476c-694c-40d9-8492-567cc8d2e2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.849115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9476c-694c-40d9-8492-567cc8d2e2b0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.849796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9476c-694c-40d9-8492-567cc8d2e2b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:52.851212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9476c-694c-40d9-8492-567cc8d2e2b0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:52.863379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:52.941531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741961-74e6-460f-aae5-457d8852f421 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:52.945809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 366/732] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:56:52 2022] POST /identity/v3/services => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:52.947458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e789fa-5abe-421b-b48a-170e54b2f586 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.947987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e789fa-5abe-421b-b48a-170e54b2f586 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.948623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e789fa-5abe-421b-b48a-170e54b2f586 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.954833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 367/733] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:52.956627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22daa6e-ca5d-4a06-9490-7228844cee00 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:52.957140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22daa6e-ca5d-4a06-9490-7228844cee00 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:52.957620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22daa6e-ca5d-4a06-9490-7228844cee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:52.981434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b22daa6e-ca5d-4a06-9490-7228844cee00 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:52.983119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b22daa6e-ca5d-4a06-9490-7228844cee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:53.021879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:53.036000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.036656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.036941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.037217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.037910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:53.038216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:53.039742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:53.066481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 367/734] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:53.071894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.080893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 368/735] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:53.177688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530839-2be9-4d36-81f5-ff1cb8b6850d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.178296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530839-2be9-4d36-81f5-ff1cb8b6850d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.178296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530839-2be9-4d36-81f5-ff1cb8b6850d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.179996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 369/736] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:56:53.185270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.185625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.186028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.190054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:53.209024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:53.209902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7db97b-8c2a-495c-9393-cc7e7e568081 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:53.210885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:53.249310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.249943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.250215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.250481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.259346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:53.262147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:53.275652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:53.277203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:53.278095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:53.282262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 368/737] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:53.283582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.307399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 370/738] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 18:56:53.398489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.403885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c305f-5520-45df-bace-9dde42ed0c12 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:53.446285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.446937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.447213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.447483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.448714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:53.449021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:53.450465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:53.459871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:53.468722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 369/739] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:53.470281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.518946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:53.524758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de87b2-eae6-4236-86da-ab86f6b394ed None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:53.562220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253d3a1-91d1-4c3a-a9c9-baa40e04980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.562903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d3a1-91d1-4c3a-a9c9-baa40e04980a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.563173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d3a1-91d1-4c3a-a9c9-baa40e04980a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.563483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253d3a1-91d1-4c3a-a9c9-baa40e04980a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.578796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253d3a1-91d1-4c3a-a9c9-baa40e04980a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7253d3a1-91d1-4c3a-a9c9-baa40e04980a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:53.583821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:53.584735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:53.586023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 371/740] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:53.586750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e3ed5-8461-42a9-8e64-f351fb3001a3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.591595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e3ed5-8461-42a9-8e64-f351fb3001a3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.591898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e3ed5-8461-42a9-8e64-f351fb3001a3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.593459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 372/741] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:53.594249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e3ed5-8461-42a9-8e64-f351fb3001a3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.613677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef89779-ad96-4588-b8cd-907fb0a2fbc1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:53.659312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.660247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.660520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.661115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.662122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=42374efb278647738c5834bdb80db3cb) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:53.662435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:53.664645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe57580-6d2e-4f96-a9b2-91a39e6bec3b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:53.682754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 370/742] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:53 2022] GET /identity/v3/role_assignments?user.id=42374efb278647738c5834bdb80db3cb&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:53.682754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52873218-f842-4b61-b29e-8432de4f101e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.682754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52873218-f842-4b61-b29e-8432de4f101e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.682754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52873218-f842-4b61-b29e-8432de4f101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.706395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-52873218-f842-4b61-b29e-8432de4f101e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:53.707332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e3ed5-8461-42a9-8e64-f351fb3001a3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:53.707841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-52873218-f842-4b61-b29e-8432de4f101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:53.747038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.747748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.748094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.748433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.749184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:53.749484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:53.750892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:53.766606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 373/743] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:53.766606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.790123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:53.807671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:53.819171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 371/744] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 18:56:53.821312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52873218-f842-4b61-b29e-8432de4f101e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:53.907313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe092-47b1-4052-91b1-268ef53a8133 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:53.955084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:53.955606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:53.955879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:53.956144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:53.956812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:53.958171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.001066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52873218-f842-4b61-b29e-8432de4f101e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.039004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.039700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.039978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.040247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.050291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 374/745] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:56:53 2022] POST /identity/v3/services => generated 240 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:56:54.061103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:54.062713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:54.079875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:54.080868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:54.081765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:54.081765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:54.081765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:54.081765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 372/746] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:54.088560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.202699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:54.242000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8ff3-8fec-4418-a8c5-43d365576961 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.251799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.280810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.281430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.281705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.281973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.282651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:54.282953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:54.285444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.300779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 375/747] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:54.309844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:54.310133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.382427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fc896-fbfb-4c4c-a062-aa4775defd74 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.415494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9e9c6-2c80-4d08-8169-759fcd563e3d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.422183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.423103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.423638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.424111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.435854 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:54.441861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:54.442811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:56:54.443747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:54.446433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 373/748] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:54.452008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.460567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.461207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.461479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.461746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.468278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:54.476413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:54.481024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:54.493617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:54.494754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:56:54.495714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:54.499401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 376/749] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:54.505126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.617563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe195464-de69-40ef-a7bd-926da6fb67d2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.631254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a23f7-f069-4ea5-9b61-f1c05509e4e4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.639635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:54.657311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.662026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.662026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.662026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.662026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:54.662026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:54.662026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.688419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.689130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.689480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.689981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.690881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:54.691261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:54.692695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.693487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 377/750] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:54.699092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.701866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 374/751] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:54.707589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.807537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:54.858324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322262fa-0a8d-400b-9a22-ed915c2c118c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.893743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.894704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.894704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.895243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:54.904270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad42b-2c8f-45b3-a485-509632e7d2df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:54.905938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:54.907517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:54.923840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:54.925024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:54.926094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:54.926094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:54.926094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:54.927479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 378/752] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:54.932941 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:54.979246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:54.979878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:54.980152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:54.980423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.000637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.002082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:55.023809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:55.025207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:55.027006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:55.027006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:55.027006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:55.027006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 375/753] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:55.032682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.074294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:55.098391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84085509-db32-42c5-8424-e800372a0be0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.109313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:55.167436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.168061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.168348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.168620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.169269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:55.169572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.175027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.199632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 379/754] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:55.203474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf883a-80ac-4e94-80aa-6131e50676de None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.205917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.289467 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.290609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.292225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.293263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.294182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:55.294839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.297060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.328046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 376/755] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:55.333809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.386791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa66054-cb8e-4df2-be74-d967df7c7839 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.421320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:55.442052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd67d5-e6a6-4f59-91aa-1156f76756b1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.476189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.477786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.478595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.480012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.484049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.485015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.486250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.489824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.491566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.499303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.502821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.506472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:55.517921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:55.518940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:55.520627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:55.534172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:55.535569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:56:55.536506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:55.542689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 380/756] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:55.544286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 377/757] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:56:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:55.546624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.555508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.611335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:55.688105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b550-cee9-42c8-bb82-02395cf3cf6b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.688632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0191885-acb9-42bd-9f53-e79705c50a4b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.722887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.723512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.723792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.724079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.724776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:55.725157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.725804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.726628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357acc4-7340-4b0b-b0ad-12939a440191 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.727039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.727521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.727983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.729530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:55.730038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.731711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.749007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:55.762575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 381/759] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:55.762955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 378/759] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Sep 7 18:56:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:55.763109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006d433-296e-4a25-b7c0-59e6dd3b3437 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.763376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006d433-296e-4a25-b7c0-59e6dd3b3437 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.763655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006d433-296e-4a25-b7c0-59e6dd3b3437 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.764798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.765232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 382/760] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:56:55.770333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.770687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.771018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.790167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:55.792037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:55.874309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 383/761] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:56:55.875323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:55.882659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69786604-ec7d-4833-b64f-80605b03e517 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:55.895483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:55.960686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:55.961766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:55.962245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:55.962755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/a500e08c87dc4e45899d7408e7b906cc/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:55.971947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=a500e08c87dc4e45899d7408e7b906cc, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:55.973662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:55.985803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076283e6-6765-4d68-bf40-4f5b03e52a52 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.020889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.021503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.021780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.022052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.076776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 379/762] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:55 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/a500e08c87dc4e45899d7408e7b906cc/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:56.093902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.095410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:56.112431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:56.156490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.222163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 384/763] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:56:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:56:56.232540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.271319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:56.324995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde72ad8-757c-46c2-a31d-19f2d249ad01 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.369362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.370101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d9c5-a9a8-441e-b78b-8ed3538411ef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.370618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.371097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.371848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.373633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.375272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:56.385980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6723caff-9da5-4f6a-bb66-d2c2fdde764a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:56.388727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:56.389734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:56:56.390591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:56.397858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 380/764] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:56.397858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69999ab-93ca-477a-9d05-7faa2a454fae None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.397858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69999ab-93ca-477a-9d05-7faa2a454fae None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.397858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69999ab-93ca-477a-9d05-7faa2a454fae None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.397858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 381/765] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:56.401477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69999ab-93ca-477a-9d05-7faa2a454fae None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.417665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.418232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.418505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.418809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.430026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.431689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7aaafeb9-c166-4c28-ba5a-a7f40e6c4aa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:56.448698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:56.449702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:56.450640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:56.450640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:56.450640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:56.450640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 385/766] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:56.455771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.455771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.455771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.471635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:56.472840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:56.545819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69999ab-93ca-477a-9d05-7faa2a454fae None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.558075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 386/767] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:56:56.559384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.573128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:56.593541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.594633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.595119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.595799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.597641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:56.598415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.601599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:56.610944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 382/768] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:56.620098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.738437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:56.750800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147fd84-58e9-4fc0-b60f-19f3450c9ca6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.804370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c8970-8414-4912-aca5-ba64ec43bdf8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.819430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.820037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.820311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.820577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.821217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:56.821511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.826393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:56.834765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 387/769] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:56.840933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.883193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.884331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.884814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.885851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.886799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:56.887308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.888989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:56.895091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 383/770] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:56.896313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:56.949769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f201f-bc66-4fa7-8d51-2e514d78e020 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:56.959293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:56.987733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:56.988366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:56.988697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:56.988916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:56.996656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:56.998312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:57.011525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:57.012429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:57.013307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:57.013742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef987ec-998c-49c5-b1ba-d01e3dbf6727 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.013817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 388/771] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:57.013817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.036793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.053358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fa29c-8c05-4f5d-8b62-69c08ce199b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.053989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fa29c-8c05-4f5d-8b62-69c08ce199b2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.054268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fa29c-8c05-4f5d-8b62-69c08ce199b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.054576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fa29c-8c05-4f5d-8b62-69c08ce199b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.067576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fa29c-8c05-4f5d-8b62-69c08ce199b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9f6fa29c-8c05-4f5d-8b62-69c08ce199b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:57.073071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:57.081493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 384/772] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:57.082591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55b39c-0a45-4ad3-a4b2-b37b35587f7b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.083531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55b39c-0a45-4ad3-a4b2-b37b35587f7b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.083531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55b39c-0a45-4ad3-a4b2-b37b35587f7b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.083531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 385/773] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:57.083531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55b39c-0a45-4ad3-a4b2-b37b35587f7b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.136802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde6b16-dfd3-4963-9aed-ec0824f9fa0b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.173265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.174286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.174822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.175109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.175835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] RBAC: Adding query filter params (interface=public, service_id=48f6408b1b67473f8d82dfd5377fcd4b, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:57.176170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.177747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.181290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55b39c-0a45-4ad3-a4b2-b37b35587f7b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.190809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 389/774] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/endpoints?service_id=48f6408b1b67473f8d82dfd5377fcd4b&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:57.190809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.196069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.225541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.226336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.226811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.227123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.228050 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:57.228364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.230315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9bd6c-73ff-4f88-bc4b-e3bb0bbc865e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.244074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 386/775] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:57.246379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.246703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.246977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.265889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:57.267336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:57.290410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.313511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ec729-901f-4fdc-a8b3-c68ba03345c8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.340421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 387/776] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:56:57.342117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.355091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.355841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.356117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.356385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.357102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:57.357402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.358772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.373616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 390/777] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:57.374822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.470992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.475680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0d83-a408-451f-b13b-4f074fa20945 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.502901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204871b2-9d0d-46b1-9a66-40ecf5d3b24b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.524907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.526053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.526539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.527275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.528469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.530589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.538290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 388/778] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:57.539353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.546914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.547568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.547887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.548210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.548946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=1da0e933ee954ba7be23a1997e4e5735) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:57.549254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.552972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.565616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 391/779] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/role_assignments?user.id=1da0e933ee954ba7be23a1997e4e5735&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:57.655459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.666938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.681356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57dac77-2b7e-4850-988e-9638f53ceef0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.732154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.733267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.733914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.734388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.749879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.751968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:57.773002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:57.773996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:57.774954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:57.774954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:57.774954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:57.774954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 389/780] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:57.777533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c769f62-6c1d-40a6-98e9-8b3467bcffb1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.787618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.825313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.852935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.854198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.854480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.854782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] PATH_INFO: `/v3/services/network` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.855377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.856802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:57.867415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:57.868458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:56:57.869852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:57.873948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 392/781] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:57.875456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:57.924179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771bf22-0ed9-4af3-9d0f-11676b952459 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.950803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:57.964064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:57.964686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:57.964968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:57.965247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:57.965894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:57.970631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:57.970631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:57.984174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2073e0-120d-4698-9fa4-fe339c1fc850 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:57.991934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 390/782] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:57.998038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:58.021762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.022375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.022679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.022952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.026018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:58.026018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.026018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.034890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 393/783] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:58.037398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:58.134138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:58.145175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf497734-0747-4210-9ea6-09040e5f99d3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:58.187016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8e8a3-b2fa-4486-bdeb-9f030871e96d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:58.205011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.205624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.205902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.206160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.206810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] RBAC: Adding query filter params (type=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:58.207106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.208457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.210480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:58.222329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 394/784] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:56:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:58.228378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:58.265497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.266171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.266646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.266919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.277416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.279150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:56:58.305346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:58.306410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:56:58.307927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:58.309721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 391/785] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:58.313085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:58.419672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af7de7-3bd1-44f7-a841-5e7df3b3cdba None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:58.457540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.458506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.459011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.459360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.460112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] RBAC: Adding query filter params (interface=public, service_id=d66585dfd52b48f99d07b0c824f818a3, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:58.460508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.462025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c149b7-f67e-4ade-959c-474493ef6d13 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.472459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 395/786] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:56:58 2022] GET /identity/v3/endpoints?service_id=d66585dfd52b48f99d07b0c824f818a3&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:58.472459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035b26-b455-4cdd-85fb-0de1d98e9049 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.472459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035b26-b455-4cdd-85fb-0de1d98e9049 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.472459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035b26-b455-4cdd-85fb-0de1d98e9049 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.472459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 396/787] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:56:58.473223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02035b26-b455-4cdd-85fb-0de1d98e9049 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:58.480255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefa05e-b860-49fb-9d9a-bfe320859f3b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:58.524765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.525775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.526253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.526746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.527742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:58.528247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.529916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d252cb-46ec-47fe-af55-cd593e6c4441 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.533311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:58.552380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 392/788] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Sep 7 18:56:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:58.554029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.554477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.554943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.576659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:56:58.578224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:56:58.660425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 393/789] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:56:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:56:58.661577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:58.684739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:58.731672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02035b26-b455-4cdd-85fb-0de1d98e9049 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:58.774927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.776434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.776849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f6a3d-2df4-4928-8ec3-0ac75d11dbac None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:58.777102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.777552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.778765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.781237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.788797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 397/790] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:56:58 2022] GET /identity/v3/services => generated 772 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:58.815846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:58.829039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:58.829972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:58.829972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:58.831461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:58.841769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:58.843408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:56:58.859877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:58.860936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:56:58.861842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:56:58.861842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:56:58.861842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:58.868375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 394/791] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:58.874540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.022923 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f9e98-e434-43e2-a6aa-445e8f6c5529 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.055972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:59.065439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.066278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.066751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:59.070498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:59.070498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:59.070498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:59.070498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:59.082455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 398/792] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:59.087664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.151718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.222658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1467a68-38b4-47d4-9e20-f2a10af14ac7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.297314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.297949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.298224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:59.298496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:59.315411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:59.316800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:56:59.334210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:59.335163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:56:59.336061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:56:59.341624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 395/793] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:56:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:56:59.347416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.363276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495c5c5-9b78-4776-a984-b6bf11f5da05 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.417899 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:59.432546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.433180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.433458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:59.435237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:59.452068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:56:59.458572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:56:59.460738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:56:59.462820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 399/794] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:56:59.471491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.512582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2170bd3-1092-46c4-bd3a-39aa9518d0aa None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.549357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.550326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.550857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:59.551354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:59.552267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:59.552856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:59.554628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:59.558321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:59.576605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 396/795] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:56:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:56:59.576605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.685488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:59.692863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466d87e-8545-433d-af20-42a2b36c6e22 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.709462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9529-aee3-4dcc-a874-5fe8b42e6633 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.763889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.766174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.766831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:59.768298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/42374efb278647738c5834bdb80db3cb/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:59.772677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=42374efb278647738c5834bdb80db3cb, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:59.775331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:59.783160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.784376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.786244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:56:59.786747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:56:59.787682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:56:59.788247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:56:59.789952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:56:59.799897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 400/796] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:56:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:56:59.805818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:56:59.867992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 397/797] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:56:59 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/42374efb278647738c5834bdb80db3cb/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:56:59.953099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c365c1e-60db-492b-925e-26e01bc12971 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:56:59.975460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:56:59.990245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:56:59.990921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:56:59.991193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.001610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.003326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:00.004848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:00.020766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:00.026153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:00.027114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:00.027114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:00.027114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:00.027114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 401/798] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:00.029086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:00.190601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a521e5-4bba-4800-a6af-72a5b8cb4c21 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:00.241640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:00.242432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.242870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.243281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.244776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:00.245232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:00.246795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c352a-b3bb-4276-873c-4b4e169fcc70 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:00.260213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 398/799] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:00.265926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:00.288064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb32b-da62-4915-beb9-1548928db968 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.288405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb32b-da62-4915-beb9-1548928db968 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.288598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb32b-da62-4915-beb9-1548928db968 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.290743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 399/800] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:00.298098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c11fc3-f92d-48fa-b5e2-18e9e579db76 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.298414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c11fc3-f92d-48fa-b5e2-18e9e579db76 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.298714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c11fc3-f92d-48fa-b5e2-18e9e579db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.323252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-57c11fc3-f92d-48fa-b5e2-18e9e579db76 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:00.324776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-57c11fc3-f92d-48fa-b5e2-18e9e579db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:00.337053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:00.392794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b585b-dd4c-4003-aaea-c71db2c0dcb0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:00.400274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 400/801] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:00.406379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0341d-c422-4071-9598-b5cb686026a4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.406379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0341d-c422-4071-9598-b5cb686026a4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.406379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0341d-c422-4071-9598-b5cb686026a4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.409037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 401/802] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:00.410975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.411115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.411446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.429681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:00.430951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:00.479802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:00.480332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.481399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.481399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.496922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:00.498368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:00.520679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:00.521672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:00.523068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:00.524997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 402/803] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:57:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:00.533453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:00.544711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 402/804] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:00.570325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:00.666986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce24c29-239d-4a54-b2ea-1a682795beed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:00.703762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:00.706003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:00.706893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.707404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.707887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.708825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:00.709358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:00.711030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:00.730861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 403/805] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:57:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:00.736906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:00.845709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:00.868779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:00.871709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca01119-6838-4bc1-a7f3-fe6ee02b8e2e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:00.908060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:00.909063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.909803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.910420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:00.924174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c2f8-36d7-4449-940a-1cb1b3030183 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:00.984836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:00.986865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:00.995888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:00.996867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:00.997800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:00.998291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.011364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:01.016181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:01.024117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 403/806] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:57:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:01.024979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.084803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.110320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 404/807] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:57:01.111542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.153355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31776-3830-48ec-93c1-007efe3c2385 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.202689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.204162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.205102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.205585 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.206686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.209013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.212134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:01.223295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:01.224499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 18:57:01.225361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:01.226108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 404/808] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:01.227245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.240113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8bcb1-c14f-4842-b70f-c349fbfbd564 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.279348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.279972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.280272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.280506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.281183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:01.281480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.282889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.297288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 405/809] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:01.297288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.342279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.407629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed34ffb-70eb-4ad8-bb1a-357a5080e253 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.445664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.446278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.446600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.446839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.447552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:01.447848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.449211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.459117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 406/810] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:01.459117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.501669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206472e-aaa1-4c67-adf0-a82c70b2af94 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.538976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.539586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.539862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.540127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.551570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.553136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-598eada6-3bc9-48ed-aa35-d6e709fa96df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:01.568705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:01.569678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:01.570636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:01.570636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:01.570636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:01.571239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23df7d-6042-4f40-9b66-20d1300ee75c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.573073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.574638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 405/811] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:01.575831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2614e6-bb00-4f1c-bbf5-8b692996adb8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.576109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2614e6-bb00-4f1c-bbf5-8b692996adb8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.576378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2614e6-bb00-4f1c-bbf5-8b692996adb8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.577882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 406/812] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:01.578455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2614e6-bb00-4f1c-bbf5-8b692996adb8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.615526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.616148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.616420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.616688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.629174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.630621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.636930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:01.647720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:01.652709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:01.653971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:01.653971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:01.653971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:01.653971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 407/813] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:01.653971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.693083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2614e6-bb00-4f1c-bbf5-8b692996adb8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.727319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.727928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.728722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.729004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.729630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:01.729931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.731333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8fdbf-efad-4863-9217-f61dcc5ce196 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.736394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 407/814] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:01.737513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.737801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.738078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.757010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:01.758116 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:01.763261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a940-4f13-4e24-8a16-4158b62d5363 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.799813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.800416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.800693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.800965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.801602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:01.801903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.803285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:01.816387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 408/815] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:01.817952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.832008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 408/816] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:01.833182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:01.855568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.890132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:01.951396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf6856-67fb-40cd-b711-3b3dfa077f73 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.958441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d6b3-edd0-4434-a339-cbd0bf468ff6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:01.991810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:01.993012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:01.993995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:01.994841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:01.995998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:01.998301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.001487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.002165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.002510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.002887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.003601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:02.003971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.005514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.018105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 409/817] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:02.023064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.104726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.110742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 409/818] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:02.112148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.132571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae32be0-2d5e-4ccf-910b-7eb8a80bdf72 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.180063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.180722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.181278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.181278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.189836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.191386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:02.204769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:02.205742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:02.206697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:02.211931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 410/819] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:02.212120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.279403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af8614-9662-4827-9cd5-7c16571ae3f1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.281389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.330479 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.331148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.331425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.331697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.339662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.341111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:02.353357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:02.354525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:02.355655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:02.356687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 410/820] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:02.359498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.364971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17653541-9c50-403d-9ce0-bd9007e595f0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.401625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.408902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.409798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.410075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.410350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] PATH_INFO: `/v3/services/network` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.414721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.414721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:02.425702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:02.426948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 18:57:02.427982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:02.429633 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 411/821] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:02.431008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.487558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b33301-39fb-488a-a69a-3443b5565a53 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.522466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.523101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.523377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.523649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.524338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:02.524645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.526621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb078f44-6440-4c41-bc53-e3696fc4e02c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.539243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 411/822] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:02.540372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d155fd1-95c2-4aa6-9502-33654a7058e3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.540713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d155fd1-95c2-4aa6-9502-33654a7058e3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.541468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d155fd1-95c2-4aa6-9502-33654a7058e3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.542822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 412/823] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:02.547399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d155fd1-95c2-4aa6-9502-33654a7058e3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.550844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5133f-fe0c-4049-97a5-f5384212b340 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.572571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.588145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.597512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.597512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.597512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.597512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:02.597512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.597512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.610587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 412/824] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:02.611437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.663793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d155fd1-95c2-4aa6-9502-33654a7058e3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.671844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.701901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.702508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.702812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.703085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.703708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] RBAC: Adding query filter params (name=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:02.704009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.705380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d351d-6c9f-4d7d-85e9-9318f7a5f99b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.709960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 413/825] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:02.710995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.711275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.711546 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.723891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2d8de-3c57-460c-8d2e-56baa86e901a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.730589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:02.731781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:02.761954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.762592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.762871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.763146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.763877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=a500e08c87dc4e45899d7408e7b906cc) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:02.764184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.765608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.772530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 413/826] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/role_assignments?user.id=a500e08c87dc4e45899d7408e7b906cc&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:02.773541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.805089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 414/827] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:02.805946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.826326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.920413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211247c2-98fb-4b14-ac7d-71e4db05fa74 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.930059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454608-e2af-40db-be37-99f196acf124 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:02.955918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:02.958102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.958806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.959087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.959408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:02.960118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] RBAC: Adding query filter params (type=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:02.960429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:02.962142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:02.973166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 415/828] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:02.974829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:02.985545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:02.987189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:02.987471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:02.987743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.006466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.008651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:03.031332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:03.032354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:03.033228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:03.033228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:03.033228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:03.036380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 414/829] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:03.043872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.136200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:03.156785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ea2b-e9c6-4a2f-a67b-0e234e78d433 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.173019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576e529-ab63-413f-a342-0ca5115bf686 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.196569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.197255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.197536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.197809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.198479 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.199878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.217136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.217790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.218079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.218354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.219037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:03.219350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.220758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.234437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 415/830] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:03.234437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.273212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:03.276496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 416/831] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:02 2022] POST /identity/v3/endpoints => generated 337 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:57:03.277329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.364038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d09a0-45a9-40e8-bc1f-24ebe81f4eea None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.417162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.418110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.418539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.419007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.432627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:03.438789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:03.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:03.441065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:03.443325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 416/832] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:03.450521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.552600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846fb1-4ae2-4781-bda1-add93d7ebf8e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.563262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:03.582841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ead1d-2938-419f-8f03-620fb450bdd0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.594046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.594851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.595227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.595596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.598792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:03.603879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.605541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:03.617308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:03.618474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:03.620171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:03.627354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 417/833] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:03.628337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.631286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.632495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.633196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.635328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.635767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:03.636078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.637465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.650590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 417/834] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:03.653705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.761790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d2b8-574f-463c-b990-abbb90a91b8b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.786327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f4916-1f81-4a57-8123-e90ede9ff8ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.802386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.803153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.803429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.803703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.804397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:03.804702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.806156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.820559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 418/835] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:03.826583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.832326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:03.833289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:03.833771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:03.834254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:03.848042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:03.849797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:03.862039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:03.871460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:03.876578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:03.882421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:03.882421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:03.882421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:03.882421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 418/836] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:03.884207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:03.909051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:03.968999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa26f6-7841-4ae5-bc66-fd2c7f3c56ed None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:03.999789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6abc40-81d1-4b76-92b1-1217d55f89ba None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:04.019161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.019748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.020024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.020295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/1da0e933ee954ba7be23a1997e4e5735/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.044556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=1da0e933ee954ba7be23a1997e4e5735, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.044773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.045308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.045583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.045852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.046562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:04.046870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.047610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.048265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.062263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 419/837] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:04.067653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:04.103281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 419/838] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:57:03 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/1da0e933ee954ba7be23a1997e4e5735/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:57:04.139109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:04.186301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76fe78-ae63-4091-82a0-8df59ee89928 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:04.226265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.227319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.227893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.228414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.239701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.239701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-370e6a51-35fa-464f-b04d-4cb5509f0b53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:04.252197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:04.253219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:04.254173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 420/839] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:04.261754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:04.376553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342578-c452-4fad-8c48-8bee28a3eec6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:04.384109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:04.411616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.412282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.412563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.412838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.413567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:04.413876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.415342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.421300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170d43c-2d2c-4447-b3e1-1716e823a21d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.421572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170d43c-2d2c-4447-b3e1-1716e823a21d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.421847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170d43c-2d2c-4447-b3e1-1716e823a21d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.423379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 421/840] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:04.428578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.428861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.429131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.440331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 420/841] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:04.448872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:04.449061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:04.449751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:04.528343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 422/842] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:04.564582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcc184-61b3-4923-b2d7-f18e034929cd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:04.600381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.601033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.601313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.601576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.633239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:04.672369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.673828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.778611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 421/843] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 18:57:04.784615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:04.798971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:04.839026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:04.880898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:04.904112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c9e0f-e76f-4c50-9244-ff0e1c4e2d81 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:04.930220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd629c-9b8b-4995-8c13-55bfc7058348 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:04.938707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.939770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.943180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.943180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.953639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.955492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.968472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:04.969501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:04.970413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:04.970888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:04.971671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:04.973224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:04.975770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:04.976719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:04.977652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:04.977652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:04.977652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:04.982413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 423/844] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:04.983608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:04.989352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:04.990367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:04.991259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:04.993995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 422/845] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:04.999480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:05.103079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b40da-3add-4012-ab13-ddd2d381ec71 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.106871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:05.131657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebdf9c-8458-4bb1-b10b-b86bebd35725 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.142891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.143807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.144243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.144654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.145428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:05.145871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.147638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.160684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:05.162080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 424/846] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:05.168300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:05.170270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.171013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:05.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.184298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 423/847] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:05.186007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:05.301821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69d11f-eca3-4993-afd7-3444df02bb59 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.319470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449281f4-32ed-4445-9b8f-cc274ffa18ad None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.342298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.343002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.343379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.343750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.352114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.354415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.363175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.364011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.364434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.364853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.365621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:05.366090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.367754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cadb3-4676-4560-950e-eb4b577d6f68 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:05.368138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:05.369587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:05.370465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 425/848] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:05.376552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:05.380746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 424/849] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:05.382756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7560876-d026-4f95-a57c-b4e14e8d6535 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.382756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7560876-d026-4f95-a57c-b4e14e8d6535 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.382756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7560876-d026-4f95-a57c-b4e14e8d6535 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.385240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 425/850] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:05.392150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.392437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.392706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.409882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:05.423949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:05.423949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:05.494477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:05.506272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc55d0-2862-48a9-a3d8-27a560d60c2e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.522658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 426/851] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:05.559142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.559765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.560040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.560308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.561011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:05.561317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.562778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.576371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 426/852] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:05.581985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:05.685408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7929a4-f5e5-4d43-9485-c9d7de3a54dc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.720513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.721387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.721862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.722325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.723247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=42374efb278647738c5834bdb80db3cb) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:05.723789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.725432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.732594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 427/853] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:05 2022] GET /identity/v3/role_assignments?user.id=42374efb278647738c5834bdb80db3cb&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:05.805826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:05.901900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:05.921199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a9cc4-c041-4c47-b600-3aa697a1bfbf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:05.957107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:05.957666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:05.957942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:05.958211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:05.958824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:05.960183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:05.969581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:05.970715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:05.971607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:05.972681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 427/854] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Sep 7 18:57:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:05.979273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:06.032986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:06.084950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34187f6b-d47d-43b8-b5af-d74896901cac None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:06.119697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:06.120310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:06.120586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:06.120857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:06.121481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:06.121781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:06.123178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0351bb5-5ef4-4c7d-b6a1-9a2f6abdbc48 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:06.132500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 428/855] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:06.133848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:06.238394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3abd35-b590-4c1f-aff1-dbb2a35882ca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:06.273456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:06.274065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:06.274344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:06.275185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:06.275799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:06.276102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:06.277462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793803f-0537-474f-af9f-55ebf5dcbc93 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:06.286721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 428/856] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:06.420144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4dd9-3bc8-4f35-9f1d-83d539c847e4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:06.420558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4dd9-3bc8-4f35-9f1d-83d539c847e4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:06.420741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4dd9-3bc8-4f35-9f1d-83d539c847e4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:06.422328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 429/857] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:06.427841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a281-9743-4911-bced-b3a05a123929 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:06.428440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a281-9743-4911-bced-b3a05a123929 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:06.428962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a281-9743-4911-bced-b3a05a123929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:06.448559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-def6a281-9743-4911-bced-b3a05a123929 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:06.450213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-def6a281-9743-4911-bced-b3a05a123929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:06.453700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:06.522334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 429/858] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:06.793324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e4dd9-3bc8-4f35-9f1d-83d539c847e4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:06.898692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e4dd9-3bc8-4f35-9f1d-83d539c847e4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:06.912734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:06.934591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:06.935301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:06.937688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:06.937688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:06.947392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:06.952457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:06.953670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:57:06.954735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:06.957117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 430/859] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:57:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:06.963116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a281-9743-4911-bced-b3a05a123929 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:07.076493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a281-9743-4911-bced-b3a05a123929 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:07.113512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:07.114096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.114377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.114685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.115360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:07.115661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:07.117023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:07.125917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 430/860] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:57:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:07.131593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:07.238439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726104a-4d0c-4541-89a4-76ca4c9e9971 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:07.275683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:07.276354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.276630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.276892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.287603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:07.289082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:07.303660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:07.304611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:07.305967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:07.305967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:07.305967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:07.305967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 431/861] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:07.310414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:07.313414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:07.418645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8805e8-fb03-4a8e-815d-8f6b80dcc024 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:07.454286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:07.455193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.455657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.456110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.457043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:07.457527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:07.459089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:07.472838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 431/862] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:07.478627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:07.508737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:07.585290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65a51-fda2-4eda-bea2-3b631f669210 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:07.621760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:07.622751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.623207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.623679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.631787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:07.633465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-392388e0-ed92-4379-ad3f-6c71d7b9b83d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:07.646324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:07.647367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:07.648310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:07.655008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 432/863] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:57:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:07.657097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:07.727552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:07.767840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00accf-cd5f-49ae-ab0c-bc7b178c5eda None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.768170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00accf-cd5f-49ae-ab0c-bc7b178c5eda None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.768393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00accf-cd5f-49ae-ab0c-bc7b178c5eda None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.769842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 433/864] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:07.772172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c88fac-72dc-4eaf-8e53-8e01e4726090 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:07.775644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f34e4-7125-4aee-a16a-e48035c95b47 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.778596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f34e4-7125-4aee-a16a-e48035c95b47 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.778596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f34e4-7125-4aee-a16a-e48035c95b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.801306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f15f34e4-7125-4aee-a16a-e48035c95b47 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:07.803222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f15f34e4-7125-4aee-a16a-e48035c95b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:07.813266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:07.813873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:07.814145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:07.814412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:07.815191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:07.815504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:07.816903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:07.837000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 432/865] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Sep 7 18:57:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:07.843216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:07.895906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 434/866] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:07.958180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c773795-5516-4692-96b9-0f58ad810f88 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:07.980567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:07.996227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.001099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31817bcd-59ba-404d-8d7e-0fb5f58e65a0 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.001099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31817bcd-59ba-404d-8d7e-0fb5f58e65a0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.001099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31817bcd-59ba-404d-8d7e-0fb5f58e65a0 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.001367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.001367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.001367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.001565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 435/867] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:08.006589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.007149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.007628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.026834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:08.028357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:08.042903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.088508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.090240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.105367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 436/868] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:08.169237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.203371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 433/869] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:57:08.209936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.240076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.314962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9437c-24a9-4ab7-9d2e-c7ec9c319ed0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.334043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eaf61d-4eb6-4d87-95a5-fd1d83bfe672 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.352666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.353660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.354226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.354798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.355667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.357418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.375282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.375911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.380411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.380411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.386964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.388382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:08.403320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:08.404314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:08.405229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:08.405229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:08.405229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:08.405229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 434/870] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:08.405229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.415493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 437/871] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:08 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:08.419591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.430853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.491082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.524577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82548-9f47-41e0-b658-a0f08e1069a0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.560552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb86e3-3e24-451d-bd73-3899d063861d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.563104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.563932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.564454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.564945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.565745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.567398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:08.577930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:08.579059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:08.579931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:08.587798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 435/872] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:08.589143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.604449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.605060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.605334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.605600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.606248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:08.606564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.607933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75723-fddd-491d-a912-24bf9dcc87c8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.621740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 438/873] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:08.622989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305ebd-183d-479f-9bcf-120db916b208 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.623263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305ebd-183d-479f-9bcf-120db916b208 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.623532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97305ebd-183d-479f-9bcf-120db916b208 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.624825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 439/874] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:08.629508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305ebd-183d-479f-9bcf-120db916b208 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.699022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.716707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85e19b-3007-47dd-bb26-bef4e85053ad None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.740014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97305ebd-183d-479f-9bcf-120db916b208 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.752384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.752956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.753233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.753505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.754137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:08.754447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.755835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ba9ab-53e9-48b4-842d-b3fc4d3372b0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.758072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.766298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 436/875] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:08.767725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.770107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.770107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.776679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.777340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.777597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.777849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.785654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.787136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.787239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:08.788361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:08.798862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:08.799766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:08.800680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 440/876] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:08.801205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.866586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 437/877] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:08.866586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:08.916617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d3a75-aa94-45b5-8091-21be8552da60 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:08.951082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:08.966110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:08.966872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:08.967198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:08.967466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:08.968102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:08.968403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:08.970312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:08.982597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 441/878] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:08.999915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bd8d-3c71-4206-98a3-7b0a77d575e3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:09.034253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:09.040617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:09.041248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:09.041526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:09.041796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:09.042505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:09.042843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:09.044297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:09.057805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 438/879] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:09.067740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:09.168993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:09.188883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829187-5503-412d-b029-98fe9840ca7b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:09.239723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:09.240330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:09.240729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:09.241142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:09.241874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=1da0e933ee954ba7be23a1997e4e5735) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:09.242331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:09.245042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e693e3-9e56-4cf1-aec8-e996cb899842 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:09.252912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 442/880] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:09 2022] GET /identity/v3/role_assignments?user.id=1da0e933ee954ba7be23a1997e4e5735&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:09.301491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e744-69d0-4d30-bb9b-5d26795054af None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:09.341887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f355-395b-4915-ac37-83d4bf00a463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:09.342524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f355-395b-4915-ac37-83d4bf00a463 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:09.342839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f355-395b-4915-ac37-83d4bf00a463 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:09.343111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f355-395b-4915-ac37-83d4bf00a463 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:09.343808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f355-395b-4915-ac37-83d4bf00a463 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:09.345223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f355-395b-4915-ac37-83d4bf00a463 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:09.449968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 439/881] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:09 2022] POST /identity/v3/services => generated 269 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:57:10.638255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f556ce-02c3-4044-8be4-ad95294429f6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:10.638837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f556ce-02c3-4044-8be4-ad95294429f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:10.639164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f556ce-02c3-4044-8be4-ad95294429f6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:10.640769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 443/882] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:10.646229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:10.646667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:10.646854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:10.665958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:10.667183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:10.715002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:10.741974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 440/883] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:11.019583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f556ce-02c3-4044-8be4-ad95294429f6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.074905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:11.129641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f556ce-02c3-4044-8be4-ad95294429f6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:11.165751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:11.166405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.166711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.166983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.167583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:11.169010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-573f7784-53ca-488a-9dcc-fdb785692547 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:11.179128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:11.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:11.184199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 444/884] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:11.191093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.290752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e5d7-b308-4e74-9e46-507d6e4edfd9 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.291057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e5d7-b308-4e74-9e46-507d6e4edfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.291215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e5d7-b308-4e74-9e46-507d6e4edfd9 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.292817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 445/885] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:11.299012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.299402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.299677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.303027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9a15f-394a-472c-aab2-65899df31202 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:11.320039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:11.321832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:11.344178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:11.344788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.345067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.345340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.346067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:11.346374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:11.347928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:11.353414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 441/886] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:11.359182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.398738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 446/887] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:11.404612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:11.468765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b263825-da39-4403-9a06-ef4d0c9edf42 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:11.508499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:11.509143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.509477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.509752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.510377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:11.510707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:11.512061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da65ad-cee4-4108-9d70-de33ce9a8b39 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:11.517387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 442/888] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:11.523276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.574960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:11.584207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2e82c-fab2-400e-aba8-76ac970288e3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.584207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2e82c-fab2-400e-aba8-76ac970288e3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.584207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2e82c-fab2-400e-aba8-76ac970288e3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.589124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 443/889] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:11.591679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9dd4-9354-43cc-aad4-e8b732a72e93 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.592570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9dd4-9354-43cc-aad4-e8b732a72e93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.592570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9dd4-9354-43cc-aad4-e8b732a72e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.613643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e53d9dd4-9354-43cc-aad4-e8b732a72e93 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:11.614969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e53d9dd4-9354-43cc-aad4-e8b732a72e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:11.638764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9bc69-14f8-4351-9372-ec93eb3f9dc2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:11.687263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:11.687982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.688162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.688417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.689304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] RBAC: Adding query filter params (interface=public, service_id=5a7bbd0c9ad34c708ae2d97b775abfd1, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:11.689422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:11.690786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f3d02-a57e-4b06-a29c-a7ecc7daca9b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:11.697802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 444/890] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:11.697802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba031ae-4e3a-4006-bcc9-af29fa7e6581 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.697802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba031ae-4e3a-4006-bcc9-af29fa7e6581 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.697802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba031ae-4e3a-4006-bcc9-af29fa7e6581 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.697802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 445/891] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:11.700146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba031ae-4e3a-4006-bcc9-af29fa7e6581 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.700254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 447/892] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:57:11 2022] GET /identity/v3/endpoints?service_id=5a7bbd0c9ad34c708ae2d97b775abfd1&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:11.703637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.703912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.704360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.725324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:11.726958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:11.742056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:11.800916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 448/893] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:11.801857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.813794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba031ae-4e3a-4006-bcc9-af29fa7e6581 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:11.846711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:11.849499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:11.850338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.850870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.851343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.852191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:11.853834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:11.911160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d3628-8c41-4ffb-a0d8-98bb7c83f41e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:11.937969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 446/894] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:11 2022] POST /identity/v3/services => generated 241 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:57:11.956484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:11.957482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:11.957963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:11.958456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:11.959358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:11.961191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:11.964864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:11.971126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 449/895] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:57:11 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 18:57:12.080105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.087928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.099410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e29172-8203-40fa-b8a8-efaf8e766039 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.142929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.144078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.144583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.145058 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.162992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:12.170284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:12.171269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:12.175395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 447/896] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Sep 7 18:57:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:12.178208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.217262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.224879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaca14b-5f15-43d0-99ef-246c427d231f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.272832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497af07-0285-4474-894d-361a11345430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.273898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497af07-0285-4474-894d-361a11345430 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.274730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497af07-0285-4474-894d-361a11345430 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.275013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497af07-0285-4474-894d-361a11345430 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.275609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497af07-0285-4474-894d-361a11345430 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.277315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497af07-0285-4474-894d-361a11345430 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6497af07-0285-4474-894d-361a11345430 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:12.290777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:12.291935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:12.292843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 450/897] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:12.297818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497af07-0285-4474-894d-361a11345430 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.302003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856451-0e8d-404f-9d96-6f32262b095c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.339832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.340555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.340892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.341230 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.341985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:12.342353 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.343802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.353560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 448/898] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:12.358951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.379947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.428036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497af07-0285-4474-894d-361a11345430 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.469510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55d576-26d6-4f0e-954c-f06676035919 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.474911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.475813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.476290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.476780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.478224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:12.478755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.480933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.492385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 451/899] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:12.493480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.514648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.515270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.515538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.515862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.527833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.529257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.531409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:12.545437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:12.549488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:12.550491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:12.550491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:12.550491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:12.552464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 449/900] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:12.553485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.627883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.638981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c721d6a-b453-4cc6-810f-077311928aac None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.664000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89b0a5-e5a6-479a-bc80-bfb39c96079d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.686506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.687383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.688336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.688759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.689808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:12.690288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.692541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.704796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 452/901] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:12.705847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.709079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.709672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.709955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.710292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.711008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:12.711302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.712785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.732872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 450/902] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:12.733813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.813458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.824103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cee10-5005-487a-b457-ce36c541a1be None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.840029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f465-570a-472f-996b-b621a3409c2c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:12.859313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.860187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.860644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.861101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.861980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] RBAC: Adding query filter params (interface=public, service_id=139b534fa5834bcb9e7b467e6508dddb, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:12.862490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.864074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.869433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:12.875831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 453/903] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/endpoints?service_id=139b534fa5834bcb9e7b467e6508dddb&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:12.877215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.881398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:12.881398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:12.881398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:12.881398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:12.889207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:12.890974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:12.906234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:12.908149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:12.910417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:12.918784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 451/904] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:12.922634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:12.988897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b97e4-b69a-4bf5-9241-abb44fff5b3c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:13.023891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:13.025016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:13.025296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:13.025568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:13.027438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:13.027534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:13.032373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb0f9e-0b91-4c0d-b0d2-f96686ef2bd6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:13.038562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 454/905] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 18:57:13.079216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:13.080228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:13.080705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:13.081298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:13.082280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:13.082866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:13.084585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:13.106852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 452/906] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Sep 7 18:57:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:13.112889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:13.141199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:13.238961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fa181-7a96-4f3c-b7f5-543d1c43b078 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:13.281088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:13.282140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:13.282697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:13.283166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:13.355098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:13.356560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:13.376706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:13.463836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 455/907] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:13 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:57:13.469328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:13.517492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:13.572419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332be858-ec67-461f-a684-56de8d3cc4b5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:13.612225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:13.612859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:13.613382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:13.613652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:13.624360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:13.625914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:13.641414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:13.642396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:13.643370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:13.643370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:13.643370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:13.646881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 453/908] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:13.648411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:13.751170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e5a10-517e-4529-bd96-695cc502211c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:13.768851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:13.786707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:13.787318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:13.787589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:13.787855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:13.788499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:13.788796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:13.790810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:13.803101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 456/909] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:13.808683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:13.908445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:13.912729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35695ac8-dad6-43a9-9679-a07531cdd7e9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:13.953812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:13.954443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:13.954741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:13.955007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:13.962010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:13.963451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:13.975099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:13.976086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:13.977099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 454/910] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:13.982708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:14.092317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbe300-1d15-4e95-96f3-0cb516ea0901 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:14.129714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:14.130360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.130666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.130942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.131663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:14.131972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:14.133527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb80d3-6fc8-4ca1-b99f-10ecee85e064 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:14.148671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 457/911] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:14.154617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:14.258929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbff520-17d8-49d0-bc5f-fded789cf4ce None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.259351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbff520-17d8-49d0-bc5f-fded789cf4ce None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.259725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbff520-17d8-49d0-bc5f-fded789cf4ce None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.262508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 458/912] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:14.267338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.267780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.268126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.271017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e8170-7d31-429c-8f6d-bdeb47745656 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:14.286827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:14.288157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:14.306427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:14.307059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.307331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.307598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.308433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] RBAC: Adding query filter params (role.id=3d46336886044db9a1c98315ee62695c, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=1da0e933ee954ba7be23a1997e4e5735) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:14.308741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:14.310270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27927a-e81a-4c4f-b425-f6dadff6a0aa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:14.316547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 455/913] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:14 2022] GET /identity/v3/role_assignments?user.id=1da0e933ee954ba7be23a1997e4e5735&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=3d46336886044db9a1c98315ee62695c => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:14.323456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:14.347815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254495ba-328f-45f8-87e8-a124d77aa722 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.348004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254495ba-328f-45f8-87e8-a124d77aa722 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.348270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254495ba-328f-45f8-87e8-a124d77aa722 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.349721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 456/914] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:14.355723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.356608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.357541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.374795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 459/915] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:14.378401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:14.379960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:14.400436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:14.455179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 457/916] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:14.654313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:14.733702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:14.757759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da646-6ed2-40af-acf2-ffaa73a6621f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:14.794593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:14.795628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.796121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.796601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.797399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:14.799022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:14.808459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:14.809599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 18:57:14.810506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:14.813296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 460/917] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:14.819953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:14.854230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:14.855129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9a1e6-fc16-4e11-a97a-5eb8257cae17 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:14.891476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:14.892545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.893050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.893577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.894443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:14.896137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:14.908074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:14.909354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:14.910200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:14.917659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 458/918] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:14.919579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:14.928483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79f8d4-b008-4265-82ec-576cfdb4534b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:14.948567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:14.964902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:14.965272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:14.965555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:14.965829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:14.966627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:14.966937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:14.968438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:14.977787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 461/919] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:14.978789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:15.039305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8062158-1b7a-4886-8cc2-76bf0ef05fae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.078172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.079241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.079710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:15.080871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.081362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.082149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:15.082634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.084156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.086342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31577d3b-8351-4dd7-828e-5dde7ae159f8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.095328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 459/920] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:15.096463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:15.123070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.123848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.124229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.124600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.125331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:15.125738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.127289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.137633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 462/921] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Sep 7 18:57:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:15.138689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:15.198256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:15.219699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba51e1-de4c-427c-8836-b33d199d5981 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.251136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27d19-02be-495c-9725-3dc9097552b3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.262239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.263280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.263816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.264303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.265170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:15.265703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.267618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.273669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 460/922] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:15.281362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:15.298153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17064f2c-ff5d-446b-afe6-30ab15062ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.298822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17064f2c-ff5d-446b-afe6-30ab15062ca1 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.299494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17064f2c-ff5d-446b-afe6-30ab15062ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.299772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17064f2c-ff5d-446b-afe6-30ab15062ca1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.300456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17064f2c-ff5d-446b-afe6-30ab15062ca1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.301840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17064f2c-ff5d-446b-afe6-30ab15062ca1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.374983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 463/923] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:15.376348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755feed-362d-4e44-aafa-cf26697a72bd None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.376789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755feed-362d-4e44-aafa-cf26697a72bd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.377076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755feed-362d-4e44-aafa-cf26697a72bd None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.378410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 464/924] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:15.388051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.388371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.388609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.394136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0035b-8a45-462b-bf16-24d4186b636b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.423973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:15.426377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:15.443601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:15.445479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.446363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.447064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.447531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.449018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] RBAC: Adding query filter params (interface=public, service_id=24d6749043a74e4f9c10aaaccc6c8c72, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:15.449789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.451414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.462995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 461/925] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:57:15 2022] GET /identity/v3/endpoints?service_id=24d6749043a74e4f9c10aaaccc6c8c72&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:15.465916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:15.471401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:15.523509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 465/926] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:15.590573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a6bf6-9065-4573-87c6-d40633fe891a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.628035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.629030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.629683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.630191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.631085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.632695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.640428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 462/927] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:57:15 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 18:57:15.802587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:15.907746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c634a-4b7c-48af-9053-7c427dab340c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:15.945088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:15.946122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:15.946774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:15.947301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:15.948098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:15.949661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:15.959195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:15.960192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 18:57:15.961084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:15.966746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 466/928] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Sep 7 18:57:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:15.970498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:16.045204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:16.084890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6e3af-d483-46db-9572-85bd6c5d5b96 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:16.120799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:16.121874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:16.122384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:16.122910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:16.123740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:16.124253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:16.125849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:16.136153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 463/929] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:16.138331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:16.155940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:16.248311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bc617-5e69-4525-b31a-b0ebb0709fb5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:16.283330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:16.284242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:16.284519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:16.284798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:16.285474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:16.285775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:16.287217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8232-1d39-46f0-b5e6-001fd975a78d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:16.296722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 467/930] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:16.298065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:16.409397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8c51-6ab1-45b7-a9a2-3834ba01a2c0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:16.440598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:16.445655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660b263-5c70-4f1e-aa86-1953cb567642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:16.446286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660b263-5c70-4f1e-aa86-1953cb567642 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:16.446581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660b263-5c70-4f1e-aa86-1953cb567642 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:16.446863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660b263-5c70-4f1e-aa86-1953cb567642 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:16.447553 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660b263-5c70-4f1e-aa86-1953cb567642 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:16.448977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660b263-5c70-4f1e-aa86-1953cb567642 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:16.546914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 464/931] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:57:16.582845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04e8e9-7b0e-44af-86fa-b8cf75b77ba2 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:16.583193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04e8e9-7b0e-44af-86fa-b8cf75b77ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:16.583396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04e8e9-7b0e-44af-86fa-b8cf75b77ba2 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:16.585163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 468/932] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:16.591233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:16.591566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:16.591850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:16.612746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:16.613986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:16.698582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 465/933] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:16.971654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04e8e9-7b0e-44af-86fa-b8cf75b77ba2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:17.051652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:17.079798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04e8e9-7b0e-44af-86fa-b8cf75b77ba2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:17.118382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:17.119463 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.119948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.120409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.131025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:17.137115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:17.138053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:17.138924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:17.141446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 469/934] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Sep 7 18:57:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:17.147267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:17.250121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3416a1-0730-4390-b33f-634f1db4d8d6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:17.264821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:17.285392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:17.286004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.286278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.286572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.287238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:17.287536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:17.288887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:17.302520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 466/935] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:57:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:17.303900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:17.413433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81976af-c8d1-41a6-aa6e-31dbee78b611 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:17.448985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:17.449653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.449944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.450224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.452163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:17.462510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:17.464213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:17.479115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:17.480261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:17.481338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:17.481338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:17.481338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:17.483636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 470/936] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:17.489187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:17.599264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2720b47-200f-429d-bc23-94a1798a5b27 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:17.636854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:17.637464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.637746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.638021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.638682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:17.638987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:17.640326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b233b73-c40b-4835-bdf5-ae542301da82 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:17.657290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 467/937] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:17.658945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:17.707825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2f9ea-974a-4e1a-b68d-ae94f0630095 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.708153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2f9ea-974a-4e1a-b68d-ae94f0630095 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.708731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2f9ea-974a-4e1a-b68d-ae94f0630095 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.710678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 468/938] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:17.716832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.717109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.717381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.738950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:17.740790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:17.777551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:17.786635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd16aa2-a114-48b9-acab-517ec480e9e4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:17.822984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 469/939] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:17.828534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:17.829201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.830628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.830914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.844205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:17.845699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-98a5462e-5b14-494e-af3d-677e3372a0b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:17.858947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:17.859984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:17.860973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 471/940] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:57:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:17.866103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:17.950960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1984c-f02f-498e-be96-05c088d5d515 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.951288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1984c-f02f-498e-be96-05c088d5d515 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.951518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1984c-f02f-498e-be96-05c088d5d515 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.952967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 472/941] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:17.958909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:17.959419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:17.959903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:17.976025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81006b38-3ab2-4996-b6ac-08dc10655c9b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:17.979470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:17.981190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:18.028613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.028613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.028613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.028613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.029026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:18.029026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.029026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.042937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.050073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 470/942] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Sep 7 18:57:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:18.058171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.074879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 473/943] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:18.084975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.096732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.173905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b83ac-dd8d-4634-adbf-de92dca26500 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.198514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98857c-4e16-41d9-8c90-d8c7e4943f3c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.223679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.225110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.225854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.226329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.243919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.245005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.245378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.245726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.246755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.247559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.248407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.249367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.251214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING keystone.server.flask.common [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:18.268267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:18.269240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:18.270124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:18.270124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:18.270124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:18.272214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 471/944] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:18.277997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.299462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.sql.core [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 18:57:18.299462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 18:57:18.299462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [parameters: {'id': '9534518846964d04bc2bac2bb1b17559', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 18:57:18.299462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81601) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9534518846964d04bc2bac2bb1b17559.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9534518846964d04bc2bac2bb1b17559. Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 18:57:18.310307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 18:57:18.311157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application yield resource Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise self.value Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application session.commit() Sep 07 18:57:18.312111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.flush() Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application compat.raise_( Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 18:57:18.313015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 18:57:18.313948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 18:57:18.314785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [parameters: {'id': '9534518846964d04bc2bac2bb1b17559', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.316798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:18.318698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9534518846964d04bc2bac2bb1b17559. Sep 07 18:57:18.320906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.323767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 474/945] 10.222.0.46 () {60 vars in 1140 bytes} [Wed Sep 7 18:57:18 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:18.329958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.364190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.368557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.394177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe6e8-6f33-41b6-abdb-4cbc12c187a2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.430907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.431509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.431783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.432054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.432677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:18.432982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.434390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.440847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc81c8-876e-454f-91b1-aaf7246b104a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.447784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 472/946] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:18.448762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.479313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.480334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.480622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.480883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.487538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.489241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:18.503328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:18.504482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.505389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 475/947] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:18.505918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.564940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6de0b-b8cf-4e57-be08-5aa9f38a9470 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.602679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.603465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.603852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.604222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.605590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.607137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.608071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.609959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:18.616948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:18.618816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:18.619658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 473/948] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:18.620457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.621176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed25582-419c-428d-83a3-c6a79afccd32 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.657049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.657820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.658153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.658490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.665960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.667529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:18.678867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:18.680001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 476/949] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:18.686394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.737638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9c0c8-8769-44a4-99a0-b5977710b9e2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.777269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.777899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.778175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.778446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.779153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:18.779454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.780893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.794595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d733ee-3529-4e18-9a67-db2e73298697 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.799801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 474/950] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:18.810982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.848813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.850429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.850817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.851160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.852408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:18.852779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.854230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.866781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 477/951] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:18.871981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:18.885681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.891793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:18.930816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789191c6-dfb9-4a5a-b2c8-14ea447894c4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.972174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:18.972898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.973179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.973452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.974159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:18.974467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:18.975959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72bd4a-7df3-4321-981a-4c85ca57602a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:18.986124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f21250-7a0c-4fc0-8010-a520dfa08c14 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:18.998615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 475/952] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:18.998615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34536-f041-4391-aa19-5c440aabb163 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:18.998615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34536-f041-4391-aa19-5c440aabb163 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:18.998615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34536-f041-4391-aa19-5c440aabb163 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:18.998615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 476/953] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:18.999109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34536-f041-4391-aa19-5c440aabb163 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:19.026109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:19.026754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.027036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.028157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.028157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:19.028331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:19.029683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681904b-7979-4cf5-94fb-5ed8c90bc7f2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:19.038624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 478/954] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:19.038624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.038624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.038624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.058178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:19.061631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:19.116784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34536-f041-4391-aa19-5c440aabb163 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:19.140048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:19.140625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 479/955] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:19.141741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:19.159002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:19.160628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:19.162389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.162389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.162389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/1da0e933ee954ba7be23a1997e4e5735/roles/3d46336886044db9a1c98315ee62695c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.165277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=1da0e933ee954ba7be23a1997e4e5735, role_id=3d46336886044db9a1c98315ee62695c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:19.166909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:19.245349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 477/956] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:57:18 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/1da0e933ee954ba7be23a1997e4e5735/roles/3d46336886044db9a1c98315ee62695c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:57:19.271086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e455a17-536b-40e0-8850-ebe6ab0c593b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:19.313318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:19.314162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.314538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.314935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.315774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:19.317351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:19.388088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 480/957] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:19.424355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:19.547359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f1d8-ad52-417e-9f43-1207b22792e9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:19.592805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:19.593440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.593717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.593987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.594576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:19.595943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:19.605225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:19.606654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:19.607738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 478/958] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:19.613884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:19.692710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:19.730209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52222a30-959e-4537-8531-632fbcc78b01 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:19.766495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:19.767380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.767807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.768268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.769103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:19.769617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:19.771203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f970614-ed79-41a2-905c-933806607c0a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:19.777212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 481/959] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:19.782877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:19.836774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:19.895572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd44bb4-4374-4f4b-a3d2-cda352e922a7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:19.930821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:19.931422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:19.931693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:19.931959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:19.932578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:19.932874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:19.934216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bcbd8-1976-40a8-93c0-f0a75391af73 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:19.939124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 479/960] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:21.124690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a849352-330c-44f5-aa69-a9567edd6e55 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.125656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a849352-330c-44f5-aa69-a9567edd6e55 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.126136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a849352-330c-44f5-aa69-a9567edd6e55 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.128104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 482/961] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:21.134011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6eb0-c6cf-469f-a091-98f2bcd80aa1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.134891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6eb0-c6cf-469f-a091-98f2bcd80aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.135401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6eb0-c6cf-469f-a091-98f2bcd80aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.155229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-802f6eb0-c6cf-469f-a091-98f2bcd80aa1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:21.156777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-802f6eb0-c6cf-469f-a091-98f2bcd80aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:21.227567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 480/962] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:21.500960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a849352-330c-44f5-aa69-a9567edd6e55 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:21.534624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823925f4-0bef-4e76-b7c3-964d889e0361 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.534933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823925f4-0bef-4e76-b7c3-964d889e0361 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.535181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823925f4-0bef-4e76-b7c3-964d889e0361 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.536960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 481/963] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:21.542249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.542530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.542837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.560847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:21.562039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:21.590621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:21.626871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a849352-330c-44f5-aa69-a9567edd6e55 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:21.641497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 482/964] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:21.666716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:21.673881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:21.674919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.675431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.675928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.676824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:21.678777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:21.733506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.sql.core [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 18:57:21.733506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 18:57:21.733506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [parameters: {'user_id': 'ce58dd0e677646c0b449d45120695f0e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 18:57:21.733506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81601) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-13c39f78-7db1-4f06-8934-e561db0716b9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Sep 07 18:57:21.741672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 18:57:21.743415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application yield resource Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise self.value Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application session.commit() Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 18:57:21.744404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.flush() Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application compat.raise_( Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 18:57:21.745335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 18:57:21.746234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 18:57:21.747127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ce58dd0e677646c0b449d45120695f0e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:21.749355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 18:57:21.751297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 18:57:21.753478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 18:57:21.753478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 18:57:21.753478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 18:57:21.753478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 18:57:21.753478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 18:57:21.753478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:21.755904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 483/965] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Sep 7 18:57:21 2022] POST /identity/v3/users => generated 160 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:21.762194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:21.786590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b10d8-c640-4f91-b6a6-2ab43ec718d6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.786590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b10d8-c640-4f91-b6a6-2ab43ec718d6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.786590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b10d8-c640-4f91-b6a6-2ab43ec718d6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.789592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 484/966] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:21.794706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.795140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.795554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.816192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:21.817980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:21.889791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cee902-ef00-4994-92f6-d6506c3126ef None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:21.894793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 485/967] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:21.924427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:21.935136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:21.935913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:21.936712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:21.937085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:21.947040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:21.949201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:21.964377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:21.965382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 18:57:21.966392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:21.969133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 483/968] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:57:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:21.974932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.003785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.065676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78a775-6ac3-46e7-afc2-a6bae0e79bc5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.093221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.097255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383710-13dd-48e1-a165-b18b063944b1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.107477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2625-8bdb-458e-90a8-845f81bf6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.108415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2625-8bdb-458e-90a8-845f81bf6f1d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.109573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2625-8bdb-458e-90a8-845f81bf6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.110035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2625-8bdb-458e-90a8-845f81bf6f1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.124952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e2625-8bdb-458e-90a8-845f81bf6f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b47e2625-8bdb-458e-90a8-845f81bf6f1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:22.133440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:22.134410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:22.136606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 486/969] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Sep 7 18:57:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:22.138290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250303a0-893d-4072-9e21-41006164ff05 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.138290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250303a0-893d-4072-9e21-41006164ff05 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.138464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250303a0-893d-4072-9e21-41006164ff05 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.139745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 487/970] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:22.144573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.150341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.150341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.150498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.150498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.150498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.150498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.151764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:22.151764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.151764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.163560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:22.167373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:22.186928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 484/971] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Sep 7 18:57:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:22.187892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.282336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 488/972] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:22.282336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.297469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.316654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058617c7-1317-47e4-a2fd-a647dc557ee4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.354683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.355310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.355683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.355961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.356640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:22.356945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.358796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.370648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 485/973] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:22.379069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.461458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732519c0-7192-417e-8606-5f71a79fdec7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.479071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.513333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.514453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.514764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.516335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.517126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.519054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:22.530932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:22.531992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:22.532889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:22.534040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 489/974] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:22.539098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.544193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b23ba5-29d9-4d93-904c-09a846db0a35 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.598628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.599270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.599542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.599809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.612828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.614282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.617229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:22.631159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:22.637157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:22.638053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:22.638053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:22.638053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:22.639074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 486/975] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:22.639854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.671543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a993543-1f6c-486b-bdb2-24129a3f4258 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.706606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.707211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.707484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.707751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.708366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:22.708664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.710008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.714844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 490/976] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:22.719561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.748843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8c1cd-4b59-407e-a7ec-7e94cdbcad05 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.760852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.782391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.783077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.783301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.783892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.784555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.786292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.818337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954bfd-bb76-48f5-99ee-a152b20b62fb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:22.847706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 487/977] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:22 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:22.848572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.868073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:22.868718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:22.869000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:22.869273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:22.869936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:22.870243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:22.871695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:22.886532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 491/978] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:22.892407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:22.915591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:22.963903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d199fa-1014-4a17-bc60-361c89324efd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.002861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:23.005011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.005640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.005911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.006249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.006930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:23.007225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.009211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.010208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec528de-70a8-4cbf-8f5a-c009323876e9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.015667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 488/979] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:23.022287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:23.055133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.055737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.056011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.056275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.064540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.065997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:23.078531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:23.079628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:23.081149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 492/980] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:57:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:23.085207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:23.141862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4a977-7e86-4062-ba25-b7178da876a7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.162235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:23.178422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.179098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.179379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.180332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.181079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] RBAC: Adding query filter params (interface=admin, service_id=24d6749043a74e4f9c10aaaccc6c8c72, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:23.181393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.182779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.188471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 489/981] 10.222.0.46 () {56 vars in 1249 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/endpoints?service_id=24d6749043a74e4f9c10aaaccc6c8c72&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:23.191358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7ccc2-662f-43d9-b4d5-d4f38947755b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.194056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:23.226218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.227686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.228033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.228303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.229027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:23.229330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.231038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.251523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 493/982] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:23.256840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:23.273152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:23.310264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552013c-6a4e-4986-a677-45ec51c9dc75 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.348176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.349082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.349365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.349642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.350255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.351694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.358500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 490/983] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 18:57:23.375573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d5539-d8b8-49cb-afee-d8984a94fa9d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.416076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.416709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.416983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.417248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.490233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.491743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.508301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:23.598767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 494/984] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 18:57:23.604753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:23.656626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:23.708919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62021bed-3004-4cb9-acf9-e2ddd0bf6940 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.743487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.744228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.744513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.744787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.756995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.758489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:23.773013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:23.774454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:23.775398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:23.775398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:23.775398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:23.778651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 491/985] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:23.780308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:23.883067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9905b9-6d29-41e2-a475-32f1ccf86a88 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:23.896872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:23.917299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:23.917986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:23.918266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:23.918534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:23.919190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:23.919491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:23.920847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:23.932611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 495/986] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:23.938682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:24.035352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:24.038884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd9973-0c90-4aa6-8e17-ffaa48880a37 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:24.072808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:24.073465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:24.073742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:24.074015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:24.080883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:24.082322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:24.095238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:24.096180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:24.097081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 492/987] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:24.102703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:24.204005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a534c-7af1-48a6-a400-d45563f60858 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:24.247648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:24.248279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:24.248557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:24.248831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:24.249545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:24.249856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:24.251365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9629380-b508-4107-ad51-ed732eefd79e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:24.264819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 496/988] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:24.270642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:24.371363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1aac6-0a4c-46c6-ac02-0f481055da82 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:24.405161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:24.405764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:24.406041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:24.406315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:24.407068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] RBAC: Adding query filter params (role.id=3d46336886044db9a1c98315ee62695c, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=1da0e933ee954ba7be23a1997e4e5735) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:24.407438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:24.409117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492087ac-adaf-40ec-8a23-9dbf768a0136 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:24.422587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 493/989] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:24 2022] GET /identity/v3/role_assignments?user.id=1da0e933ee954ba7be23a1997e4e5735&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=3d46336886044db9a1c98315ee62695c => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:25.036551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5acd7-d02f-4243-be32-619a4db839b3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.036926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5acd7-d02f-4243-be32-619a4db839b3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.037434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5acd7-d02f-4243-be32-619a4db839b3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.039392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 497/990] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:25.045840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f16c5f-dc90-42f2-8d14-ec1351a909c3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.046130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f16c5f-dc90-42f2-8d14-ec1351a909c3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.046592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f16c5f-dc90-42f2-8d14-ec1351a909c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.065961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f6f16c5f-dc90-42f2-8d14-ec1351a909c3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:25.067179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6f16c5f-dc90-42f2-8d14-ec1351a909c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:25.084935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:25.135459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 494/991] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:25.400020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5acd7-d02f-4243-be32-619a4db839b3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:25.439483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:25.508272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5acd7-d02f-4243-be32-619a4db839b3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:25.538290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975d690-98f7-43f3-8b42-04c4c49d836c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.539348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975d690-98f7-43f3-8b42-04c4c49d836c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.539435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975d690-98f7-43f3-8b42-04c4c49d836c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.540841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 495/992] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:25.543095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:25.543724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.544002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.544515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.544922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:25.545576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.545860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.546138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.546420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:25.555719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:25.556734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:25.557586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:25.561663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 498/993] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:25.562727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:25.565334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:25.566732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:25.637872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 496/994] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:25.667619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:25.671477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6835219-beb4-492b-a86c-025537d4e05b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:25.705448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:25.706049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.706323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.706617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.707238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:25.707537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:25.708895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:25.713787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 499/995] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:25.719419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:25.803789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:25.819540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26945fc-a393-45eb-b530-9852f9815a15 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:25.854908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:25.855555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:25.855832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:25.856126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:25.856745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:25.857041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:25.858385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:25.863237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 497/996] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:25.869066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:25.919880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:25.969035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9972605-8a3c-444a-ab76-8f47c88fa54a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:26.005325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:26.005858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.006125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.006392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.007138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] RBAC: Adding query filter params (interface=public, service_id=a992a930adf544fca919645c5a2192d1, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:26.007654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:26.008858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:26.018515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 500/997] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:57:25 2022] GET /identity/v3/endpoints?service_id=a992a930adf544fca919645c5a2192d1&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:26.019520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:26.030580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ff8a-7a2c-4b67-895b-2f4450357476 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:26.047654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:26.068180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:26.069083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.069362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.069648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.070221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:26.071623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:26.080948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:26.081944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:26.082818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 498/998] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Sep 7 18:57:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:26.088394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:26.104470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:26.130124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6307b-3a70-403f-9789-3b34d63da2fe None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:26.167065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:26.168115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.168115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.168309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.168886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:26.170305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:26.176858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 501/999] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:57:26 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 18:57:26.202162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782889-48c9-4b65-b77d-c19f388bc4d3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:26.247462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:26.248173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.248456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.248727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.249382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:26.249684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:26.251231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:26.256988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 499/1000] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:26.262731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:26.365963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:26.375582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb59bd9-8d4b-47c0-a40d-49a4c01cfceb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:26.410653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:26.411272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.411578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.411787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.412407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:26.412706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:26.414101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8587565-59cc-4a04-850f-557ac2bd8a97 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:26.419062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 502/1001] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Sep 7 18:57:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:26.424691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:26.502095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:26.530849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34af2b-e1fa-4c77-9bfd-85dc8f26bf58 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:26.566262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:26.567103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.567189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.567459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.568095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:26.569449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:26.630092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fb08-39f0-4a02-a261-c497913bb778 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.630377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fb08-39f0-4a02-a261-c497913bb778 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.630691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fb08-39f0-4a02-a261-c497913bb778 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.632037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 503/1002] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:26.637713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:26.637996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:26.638310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:26.643461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 500/1003] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:57:26.657785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:26.659076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:26.751657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 504/1004] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:26 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:27.036905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:27.142276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518386e-cefd-456a-a3f3-b162e6477eac None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:27.176093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:27.176720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:27.176996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:27.177266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:27.177846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:27.179249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:27.181747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:27.188687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:27.196855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:27.197900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 501/1005] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Sep 7 18:57:27 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:27.198795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:27.288066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:27.309749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335414f-a11a-4a62-baa5-a6d8c3680209 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:27.344991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:27.345597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:27.345873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:27.346140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:27.346781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:27.347081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:27.348435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec611e9-f7f5-41c6-b3a0-18e3d6a2c0e6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:27.358515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 505/1006] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:27.359738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:27.460175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94311-f1cd-4208-a6d9-88364d57c28d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:27.494411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:27.495044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:27.495318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:27.495587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:27.496206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:27.496500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:27.497845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478591b3-8958-4aaf-b571-161ebcb049e2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:27.502402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 502/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:28.372749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9dfe9-dbb9-40ce-b8a2-776a9b65dcf9 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:28.373266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9dfe9-dbb9-40ce-b8a2-776a9b65dcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:28.373373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9dfe9-dbb9-40ce-b8a2-776a9b65dcf9 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:28.374964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 506/1008] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:28.380445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cc565-0b3a-453e-b8a7-93594c513a41 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:28.380849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cc565-0b3a-453e-b8a7-93594c513a41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:28.381048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cc565-0b3a-453e-b8a7-93594c513a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:28.399701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a85cc565-0b3a-453e-b8a7-93594c513a41 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:28.400890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a85cc565-0b3a-453e-b8a7-93594c513a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:28.459952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:28.468878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 503/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:28.737726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9dfe9-dbb9-40ce-b8a2-776a9b65dcf9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:28.842499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9dfe9-dbb9-40ce-b8a2-776a9b65dcf9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:28.845391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165339e-8936-4e4b-89f9-b79ce097e60c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:28.845675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165339e-8936-4e4b-89f9-b79ce097e60c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:28.845949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165339e-8936-4e4b-89f9-b79ce097e60c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:28.847500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 504/1010] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:28.852894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:28.853179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:28.853458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:28.872145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:28.873327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:28.881483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:28.882110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:28.882410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:28.882695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:28.883276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:28.884626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:28.893295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:28.894338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:28.895248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 507/1011] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:28.900619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:28.911568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:28.947341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 505/1012] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:29.010644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0432c-ddc0-4291-8ed2-48cfa48de078 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:29.046627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:29.047238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.047511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.047779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.048423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:29.048726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:29.050083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:29.059682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 508/1013] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:29.060714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:29.162289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa46ad-0ac4-4e05-bb7e-0e06e7c18c80 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:29.186426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:29.198128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:29.198764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.199045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.199314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.199934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:29.200231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:29.201615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:29.210860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 506/1014] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Sep 7 18:57:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:29.215656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:29.220392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:29.297255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:29.328370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f86551-d772-462c-9971-a43ea9f79e8e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:29.336204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec5351-bd69-4988-975b-b7bc8bbae481 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:29.365886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61c06-6603-4c78-bd32-870469b46be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:29.367291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61c06-6603-4c78-bd32-870469b46be8 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.367571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61c06-6603-4c78-bd32-870469b46be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.367842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61c06-6603-4c78-bd32-870469b46be8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.368504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61c06-6603-4c78-bd32-870469b46be8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:29.369870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61c06-6603-4c78-bd32-870469b46be8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:29.374071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:29.374731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.375012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.375290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.376406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:29.377815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:29.384706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 507/1015] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:29.385605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:29.440758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:29.441894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 509/1016] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:29.490063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22777584-23b5-40ef-bdb4-fecd3b570767 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:29.535627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5da8a4-8e11-4f57-8a02-f3746f54d1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:29.536260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5da8a4-8e11-4f57-8a02-f3746f54d1a4 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.536536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5da8a4-8e11-4f57-8a02-f3746f54d1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.536814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5da8a4-8e11-4f57-8a02-f3746f54d1a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.537557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5da8a4-8e11-4f57-8a02-f3746f54d1a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:29.539151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5da8a4-8e11-4f57-8a02-f3746f54d1a4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:29.627861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 508/1017] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:29 2022] POST /identity/v3/projects => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:29.676084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fecf03-5b62-4769-935a-0a821ee7bd17 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.676641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fecf03-5b62-4769-935a-0a821ee7bd17 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.676919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fecf03-5b62-4769-935a-0a821ee7bd17 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.678287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 510/1018] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:29.685462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b2a87-1193-47d6-b070-9d04ddb2a3cc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:29.685768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b2a87-1193-47d6-b070-9d04ddb2a3cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:29.686100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b2a87-1193-47d6-b070-9d04ddb2a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:29.706971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-559b2a87-1193-47d6-b070-9d04ddb2a3cc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:29.708205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-559b2a87-1193-47d6-b070-9d04ddb2a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:29.784925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 509/1019] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:29 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:30.055112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fecf03-5b62-4769-935a-0a821ee7bd17 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:30.157483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fecf03-5b62-4769-935a-0a821ee7bd17 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:30.185559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:30.193525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1cd4-fc2a-425a-9f04-bfdb3975ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:30.194128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1cd4-fc2a-425a-9f04-bfdb3975ed25 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:30.200761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1cd4-fc2a-425a-9f04-bfdb3975ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:30.200761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1cd4-fc2a-425a-9f04-bfdb3975ed25 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:30.200761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c1cd4-fc2a-425a-9f04-bfdb3975ed25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:30.200761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c1cd4-fc2a-425a-9f04-bfdb3975ed25 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:30.282160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 511/1020] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:30 2022] POST /identity/v3/services => generated 242 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:31.771121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c3ff-6d21-4ee8-94fe-8c8ac4e28eba None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:31.771669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c3ff-6d21-4ee8-94fe-8c8ac4e28eba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:31.771669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c3ff-6d21-4ee8-94fe-8c8ac4e28eba None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:31.773704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 510/1021] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:31.778861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:31.779240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:31.779420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:31.797529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:31.798738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:31.844483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3439e8-5b31-49ea-82c0-12474881e07f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:31.844752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3439e8-5b31-49ea-82c0-12474881e07f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:31.845036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3439e8-5b31-49ea-82c0-12474881e07f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:31.846480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 511/1022] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:31.851248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:31.851527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:31.851797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:31.870046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:31.871249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:31.876211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 512/1023] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:31.900129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:31.941638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 512/1024] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:32.150138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.212154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.257495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1432-3de6-4f40-b3e6-15b12db52ed7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.272193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:32.293176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.293982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.294332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.294963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.305430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:32.310383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:32.311352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:32.312196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 513/1025] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Sep 7 18:57:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:32.318130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.323154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a89e3-31fd-46d4-8184-bea6f904644c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.361686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.362815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.363296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.363765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.365172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:32.367117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:32.369299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 513/1026] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:32.374623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.409486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:32.431144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392ab9-a5ff-4517-bdfc-757e8efe0ebd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.466781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.467407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.467679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.467944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.468595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:32.468890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:32.470242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e613d-1f83-4340-bbbe-92eb482cd848 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:32.478325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 514/1027] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:57:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:32.479556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021c698-7794-4e31-aef6-238f94b667f8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.480032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021c698-7794-4e31-aef6-238f94b667f8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.480309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021c698-7794-4e31-aef6-238f94b667f8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.481678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 515/1028] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:32.483893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021c698-7794-4e31-aef6-238f94b667f8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.484914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c105a08-1f22-4b68-b789-b2c6a3f6bf48 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.519860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:32.520669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb6795-726a-433e-be67-4e5be2f664d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.521098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb6795-726a-433e-be67-4e5be2f664d4 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.521357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb6795-726a-433e-be67-4e5be2f664d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.521615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb6795-726a-433e-be67-4e5be2f664d4 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.522289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb6795-726a-433e-be67-4e5be2f664d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:32.524401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb6795-726a-433e-be67-4e5be2f664d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:32.584172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021c698-7794-4e31-aef6-238f94b667f8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.612741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 514/1029] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:57:32 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:57:32.613768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.614049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.614317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.624572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.625278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.625580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.625845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.642841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:32.644316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:32.646373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:32.647651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:32.660504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:32.661684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:32.662770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:32.662770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:32.662770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:32.664361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 516/1030] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:32.670746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.734602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 515/1031] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:32.780600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:32.794913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b595e-5a5f-4135-b7c8-646d4a2626b5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.831029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.831641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.831920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.832191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:32.832831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:32.833138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:32.834505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:32.846696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 517/1032] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:32.852074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:32.876239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:32.956826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabcf43-87a6-4a59-b954-d68e6a2cd486 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:32.992234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:32.992867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:32.993146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:32.993417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.000492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.001939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.005184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:33.014587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:33.015548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:33.016539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 516/1033] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:57:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:33.021615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.108620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.121288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe442f-5861-4861-8dfb-584d1ba02521 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.130849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90bb59-bb5f-4544-84be-1a75b8003df0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.158877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.159508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.159788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.160061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] PATH_INFO: `/v3/services/image` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.160645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.162012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.168807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.169413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.169691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.169963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.170725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:33.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:33.171846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:33.175536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:33.176300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.176375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 518/1034] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:33.176452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.177811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.191289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 517/1035] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:33.197143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.198062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.293723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66face9e-24da-4f6e-8581-25efcf6cc399 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.308164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9845b4-bb54-4d3d-bf97-d660d5ee9eb8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.326613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.327225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.327501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.327774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.328393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] RBAC: Adding query filter params (name=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:33.328710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.330064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.335412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 519/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:33.340984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.352611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.353240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.353518 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.353792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.357547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.426730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.428772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.430211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.452159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e188b-bbaf-42b4-90d5-81f9208fa402 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.494922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.495445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.495721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.496177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.496638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:33.497025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.498287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.503932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 520/1037] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:33.509691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.545243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 518/1038] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 18:57:33.546523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.588092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.622007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7a056-bc61-45b9-a98c-24a6516eb3d1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.657644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.658864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.659326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.659917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.660909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] RBAC: Adding query filter params (interface=public, service_id=5473e59e197a47978ab1e0d2d5b9ecb5, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:33.661397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.662990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.663495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c550-eb21-4d23-ae40-095aa96444fa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.673481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 521/1039] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/endpoints?service_id=5473e59e197a47978ab1e0d2d5b9ecb5&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:33.674733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.679290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.699344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.699972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.700251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.700534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.711785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.713202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:33.727967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:33.729097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:33.730186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:33.730186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:33.730186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:33.730186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 519/1040] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:33.734596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.785178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b8d58-1751-4e59-9b46-fe26a113f053 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.821641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.822500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.822950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.823387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.824123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.825657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.828980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:33.832918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 522/1041] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 18:57:33.841504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e6530-fed1-4a26-b0b1-a3fcfe1a489e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:33.879717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:33.880889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:33.880889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:33.880889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:33.881520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:33.882349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:33.883761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:33.905966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 520/1042] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:33.907372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:33.982182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:34.018878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6c15f-25f1-4e77-a97f-565f99b6ddec None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:34.055759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:34.056313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:34.056592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:34.056863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:34.063580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:34.065017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:34.076733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:34.077632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:34.079173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 523/1043] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:34.083741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:34.188891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015398d3-9cd5-4cea-af3b-52a7b0db81d9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:34.226666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:34.227275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:34.227547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:34.227815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:34.228501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:34.228800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:34.230210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13364b3-7440-40b3-8b79-3fa50d8667c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:34.251902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 521/1044] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:34.253102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:34.357111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8050708-5496-47f8-aff7-38b4a81e4d1b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:34.387722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:34.395989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:34.396588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:34.396862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:34.397133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:34.398007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] RBAC: Adding query filter params (role.id=3d46336886044db9a1c98315ee62695c, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=a500e08c87dc4e45899d7408e7b906cc) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:34.398310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:34.399764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cdaf3-6da0-4bd3-81de-45dd855acb0e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:34.414582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 524/1045] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:34 2022] GET /identity/v3/role_assignments?user.id=a500e08c87dc4e45899d7408e7b906cc&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=3d46336886044db9a1c98315ee62695c => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:34.858714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc2200-5a85-4ca1-93b3-ae1ff592da61 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:34.859064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc2200-5a85-4ca1-93b3-ae1ff592da61 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:34.859282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc2200-5a85-4ca1-93b3-ae1ff592da61 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:34.861066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 522/1046] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:34.866867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:34.867318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:34.867490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:34.885956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:34.887168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:34.954596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 525/1047] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:35.223340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc2200-5a85-4ca1-93b3-ae1ff592da61 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:35.236489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:35.331711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc2200-5a85-4ca1-93b3-ae1ff592da61 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:35.364908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:35.365528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:35.365803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:35.366100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:35.376674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:35.381412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:35.383192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:35.384091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:35.386871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 523/1048] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:57:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:35.388962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:35.487520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ccf20-bbd9-42a1-be63-ca1734e970e2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:35.510627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:35.523041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:35.523650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:35.523928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:35.524318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:35.524897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:35.525196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:35.526600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:35.539660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 526/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:57:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:35.541013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:35.608388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:35.643910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f6ff-aef8-425a-880a-b6b4112402e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:35.678974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:35.679609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:35.679887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:35.680157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] PATH_INFO: `/v3/users/441537905d6b4e22a018cbdfe8273679` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:35.693394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] RBAC: Authorizing `identity:get_user(user_id=441537905d6b4e22a018cbdfe8273679)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:35.694882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:35.701123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 524/1050] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:35 2022] GET /identity/v3/users/441537905d6b4e22a018cbdfe8273679 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:35.702243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:35.802200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89af2a-3935-4195-abf8-b5d6eb53c9d5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:35.834572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eacea7-2025-4f8b-85c7-279da29a00f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:35.835680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eacea7-2025-4f8b-85c7-279da29a00f7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:35.835957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eacea7-2025-4f8b-85c7-279da29a00f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:35.836228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eacea7-2025-4f8b-85c7-279da29a00f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:35.888051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:35.905057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eacea7-2025-4f8b-85c7-279da29a00f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:35.906542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eacea7-2025-4f8b-85c7-279da29a00f7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:36.015979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 527/1051] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:57:36.017270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.037809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbdd61-53d9-499d-9fcd-0b4fbfcfc455 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.038115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbdd61-53d9-499d-9fcd-0b4fbfcfc455 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.038374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbdd61-53d9-499d-9fcd-0b4fbfcfc455 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.039882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 528/1052] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:36.044865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.045153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.045433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.063641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:36.064759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:36.122528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138210-0b4d-4bcd-9c5a-d3a0af411c9c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:36.136844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 529/1053] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:36.150696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:36.159380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:36.160007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.160283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.160553 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.161669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] RBAC: Authorizing `identity:get_project(project_id=4c058a9e00ce4e64bda849628bbbc548)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:36.163625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:36.169965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 525/1054] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:57:36 2022] GET /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:36.171955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.188356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:36.283408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5eb1e-3c49-4ba0-b825-d3a9a3e6dd29 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:36.318146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:36.318710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.318988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.319254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.319969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=4c058a9e00ce4e64bda849628bbbc548, user.id=441537905d6b4e22a018cbdfe8273679) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:36.320277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:36.321696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e1edd-1798-42c1-8329-5aaeb59ed51c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:36.335092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 530/1055] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:36 2022] GET /identity/v3/role_assignments?user.id=441537905d6b4e22a018cbdfe8273679&scope.project.id=4c058a9e00ce4e64bda849628bbbc548&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:36.408154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.528032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272eb41-9bd5-4321-8e5e-e7fae41efbc7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:36.549905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c4e7-068d-416f-a20a-c59e0778c6a7 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.550217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c4e7-068d-416f-a20a-c59e0778c6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.550461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c4e7-068d-416f-a20a-c59e0778c6a7 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.551894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 531/1056] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:36.557179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.557476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.557748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.566254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:36.567140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.567568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.567989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.568770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:36.570863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:36.577144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:36.578317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:36.582617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:36.583712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 18:57:36.584581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:36.586704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 526/1057] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Sep 7 18:57:36 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:36.590610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.653532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 532/1058] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:36.658258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:36.702206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f0bdb-4f5e-4357-b39f-d8c7ea449f9e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:36.736554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:36.737159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.737435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.737703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.738332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] RBAC: Adding query filter params (name=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:36.738660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:36.740013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:36.745884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 527/1059] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:36.751558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.762373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:36.854614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69896d80-f926-4c6b-8a06-f3332f94f283 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:36.888758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:36.889382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:36.889634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:36.889906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:36.890529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:36.890859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:36.892225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:36.897299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 533/1060] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:36.902937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.924005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:36.994779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:37.008121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41fb94-efa4-4af6-8e75-12d0881ede15 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.027784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d7c4-e326-4d1b-ae7b-a1229ba28eb4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.032044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:37.044160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.044777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.045052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.045323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.046006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:37.047439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:37.068256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.068952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.069733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.070009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.080112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:37.084923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:37.085921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:37.086804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 534/1061] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Sep 7 18:57:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:37.092164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:37.117296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 528/1062] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 18:57:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:37.216495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cb15-0242-43b9-9069-2a83ea23ede0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.252348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:37.264909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.265642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.266005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.266236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.267027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:37.267331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:37.268789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:37.283389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 535/1063] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:57:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:37.284664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:37.395738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fcf74-8060-404e-aa74-b35b1bfdd8ea None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.429926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.430740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.430898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.431133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.441758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:37.443285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:37.457246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:37.458182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:37.463876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:37.463876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:37.463876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:37.463876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 529/1064] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:37.464439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:37.566252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72030179-5215-44aa-a797-1c035e2adf23 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.599846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.600972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.601251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.601526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.602163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:37.602469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:37.603865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:37.616605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 536/1065] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:37.622200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:37.654844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:37.726506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc70726-c9d1-4adf-a18a-23176f37c7ad None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.759690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.760315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.760593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.760866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.768138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:37.769559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:37.780964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:37.781942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:37.783551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:37.786869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 530/1066] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:57:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:37.788732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:37.839499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:37.889966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f67f51-5507-484f-abe0-72fdca71f9aa None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:37.925009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:37.925545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:37.925821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:37.926092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:37.928633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:37.928949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:37.930381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:37.952452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 537/1067] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:57:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:37.953864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:38.048549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:38.059296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f877b41-2bad-4cfa-8c4f-4bed9cd953b3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:38.093363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:38.093992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.094271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.094567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.105339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:38.106926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:38.121428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:38.122369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:38.127509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:38.127509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:38.127509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:38.127509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 531/1068] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:38.129466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:38.238110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40553a57-d1e7-442e-805a-d961572ca1c4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:38.248350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:38.273133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:38.273740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.274018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.274292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.274961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:38.275268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:38.276629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:38.293076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 538/1069] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:38.294312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:38.401397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8a38-230a-4ef4-82e9-4812c52a0326 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:38.434755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:38.435400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.435658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.435929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.442691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:38.444649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-58f827d8-3a6c-4d3b-bf6a-345ca418fb03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:38.456544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:38.457413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:38.463381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 532/1070] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:38.465494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:38.469445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0056e6-6ecc-4230-9b7b-cf4fa5cda924 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.469731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0056e6-6ecc-4230-9b7b-cf4fa5cda924 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.470009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0056e6-6ecc-4230-9b7b-cf4fa5cda924 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.471622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 533/1071] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:38.476117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.476396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.476663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.484335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:38.496763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:38.497883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:38.567513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 534/1072] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:38.580603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f2a3e-167c-47b0-a4cb-ed62ad89050c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:38.613853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:38.614457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.614768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.615040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.616232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:38.616535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:38.618475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:38.632541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 539/1073] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:38.638768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:38.738926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa370d-8733-4e6d-b144-901d7a92d224 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:38.769113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:38.775243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:38.781836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:38.781836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:38.781836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/a500e08c87dc4e45899d7408e7b906cc/roles/3d46336886044db9a1c98315ee62695c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:38.781836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=a500e08c87dc4e45899d7408e7b906cc, role_id=3d46336886044db9a1c98315ee62695c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:38.781836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:38.840908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:38.859067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 535/1074] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:57:38 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/a500e08c87dc4e45899d7408e7b906cc/roles/3d46336886044db9a1c98315ee62695c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:57:38.884055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:38.964437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ac5f5-25e3-4456-9e52-dbef35674439 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:39.001689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:39.002337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.002642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.002914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.014391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:39.019318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:39.020248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:39.021232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:39.022948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 540/1075] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:57:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:39.026677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:39.132454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd479cb-a497-4d7b-acab-51f097ed66eb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:39.167188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:39.167797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.168084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.168352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.169094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:39.169400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:39.170772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d6f64-ebd2-4307-ad61-a4a114705e1c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:39.179569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 536/1076] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:57:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:39.185322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:39.286780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:39.299671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd643df6-4370-4aa4-a135-0324f8ab5ccf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:39.335683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076be9-8c2a-42e8-bc83-5fbaff0b703d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.335940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076be9-8c2a-42e8-bc83-5fbaff0b703d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.336331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076be9-8c2a-42e8-bc83-5fbaff0b703d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.337135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:39.337767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.338149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.338253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 537/1077] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:39.338345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] PATH_INFO: `/v3/users/441537905d6b4e22a018cbdfe8273679` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.339505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] RBAC: Authorizing `identity:get_user(user_id=441537905d6b4e22a018cbdfe8273679)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:39.341131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:39.346603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 541/1078] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:39 2022] GET /identity/v3/users/441537905d6b4e22a018cbdfe8273679 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:39.346781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74c623-0008-48e6-8767-6211092451ad None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.346781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74c623-0008-48e6-8767-6211092451ad None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.346781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74c623-0008-48e6-8767-6211092451ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.349414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:39.364367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0a74c623-0008-48e6-8767-6211092451ad None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:39.365799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a74c623-0008-48e6-8767-6211092451ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:39.422829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:39.442593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 538/1079] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:39.458248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7414e3-8fcb-410b-a96f-bf220362752c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:39.494043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:39.494701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.494981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.495254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.496370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] RBAC: Authorizing `identity:get_project(project_id=4c058a9e00ce4e64bda849628bbbc548)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:39.497802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:39.504370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 542/1080] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:57:39 2022] GET /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:39.505565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74c623-0008-48e6-8767-6211092451ad None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:39.607212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74c623-0008-48e6-8767-6211092451ad None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:39.641178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:39.641886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.642169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.642440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548/users/441537905d6b4e22a018cbdfe8273679/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.644364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c058a9e00ce4e64bda849628bbbc548, user_id=441537905d6b4e22a018cbdfe8273679, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:39.645775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:39.715530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:39.743514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:39.745440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 539/1081] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:57:39 2022] PUT /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548/users/441537905d6b4e22a018cbdfe8273679/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:57:39.836877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6543f8-932b-4f0a-8660-0a732e8a20df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:39.874990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:39.875515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:39.875792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:39.876065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:39.951113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:39.952620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:39.977290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:40.053979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 543/1082] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 18:57:40.059716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:40.107932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:40.162239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1e1f2-11ff-4761-81b4-a5f36e4fb161 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:40.199079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:40.199710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:40.199981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:40.200255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:40.213618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:40.215090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2da93f97-d24a-4368-956e-916d089cf45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:40.230659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:40.231610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:40.232515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:40.232515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:40.232515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:40.238439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 540/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:40.238596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:40.340526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cb490-aadf-4617-b787-d4b19e850490 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:40.364130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:40.374929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:40.375536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:40.377033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:40.377033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:40.377033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:40.377033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:40.378345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc6bd0-e2a3-4514-81d2-22e55d6b97f5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:40.398580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 544/1084] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:41.049603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34e91-e45d-4b1d-b55b-d71dc17002f4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.050129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34e91-e45d-4b1d-b55b-d71dc17002f4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.050129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34e91-e45d-4b1d-b55b-d71dc17002f4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.051504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 541/1085] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:41.057443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.057834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.057982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.076384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:41.077556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:41.141850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 545/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:41.408781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34e91-e45d-4b1d-b55b-d71dc17002f4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:41.506622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:41.513303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34e91-e45d-4b1d-b55b-d71dc17002f4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:41.547416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:41.548041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.548317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.548589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.559013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:41.563801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:41.564759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:41.565628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 542/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Sep 7 18:57:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:41.571634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:41.669734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502881-0f3a-41e6-ada7-b2bc51895c4f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:41.683185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:41.702802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:41.703565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.704066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.704341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.705005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:41.705312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:41.706719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:41.719874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 546/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:57:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:41.721078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:41.823940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1ef0-8707-473e-9aa1-a533c0694a5e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:41.857767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:41.858384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.858697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.858973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.869346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:41.870800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a3c80c72-851e-462e-8dd7-738fdcd0c96d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:41.885161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:41.886306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:41.887436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:41.887436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:41.887436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:41.887797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 543/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:57:41.891968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:41.912338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402a61-ae5e-4cf1-acc1-52fcb89b70f0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.912618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402a61-ae5e-4cf1-acc1-52fcb89b70f0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.912895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402a61-ae5e-4cf1-acc1-52fcb89b70f0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.914198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 544/1090] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:57:41.919146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:41.919427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:41.919701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:41.937516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:41.938668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:41.940595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:41.992217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631f4c4-97cf-47e8-ac1d-002612b67e51 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.005297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 545/1091] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:42.025589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.026190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.026468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.026774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.027413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:42.027715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.029077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.040771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 547/1092] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:42.046296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.145509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530a35f-9cd1-4db3-b303-3f49b2b8b18c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.178806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.179425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.179703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.179975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.186726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.188196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.189834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:57:42.201196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:42.203978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:42.204952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 546/1093] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:42.207993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.248387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.276023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.313693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c3c8-fe41-470b-a84a-8262f71d5cc8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.347813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.348419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.348695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.348966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.349686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:42.350623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.352055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.369458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 548/1094] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:42.375189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.377236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd99472-c243-4478-b5a8-5266bf34b88f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.418231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.419178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.419674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.420141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.433054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:57:42.437793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:42.438981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:57:42.439877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 547/1095] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:42.444807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.468036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.475437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7396e6-5a9e-4e19-bde6-14e90b02579d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.503035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.508997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.509597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.509873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.510159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.545683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad2722-7809-4c3d-a2ca-0ba609913445 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.576376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.577719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.577845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.578312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.578611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.578882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.579535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:42.579831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.581151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.589966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 548/1096] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:42.595886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.673607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.681515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 549/1097] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 18:57:42.687388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.694168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96488596-ae8e-4689-8518-5696e5ad2546 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.714511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.728809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.729437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.729717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.729996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] PATH_INFO: `/v3/users/441537905d6b4e22a018cbdfe8273679` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.731141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] RBAC: Authorizing `identity:get_user(user_id=441537905d6b4e22a018cbdfe8273679)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.732564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.734137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 549/1098] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/users/441537905d6b4e22a018cbdfe8273679 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:42.739840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.784046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166e879-d133-4894-8bab-68ecab08260f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.817551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.818170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.818470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.818773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.828421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.829863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.837934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864c775-a08a-483b-a07e-d1be933a522c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:57:42.843890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:42.844934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:57:42.846452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:57:42.846452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:57:42.846452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:42.846452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 550/1099] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:42.851162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:42.872804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.873409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.873686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.873956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.895691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.929605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:42.941720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.943247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.950226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec9cb-2834-4e92-a520-17044ba87f95 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:42.982883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:42.983472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:42.983751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:42.984025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:42.984650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:42.984947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:42.986288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:42.998360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 551/1100] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:43.003826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:43.043950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 550/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 18:57:43.049689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:43.107510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72063710-1043-4cc8-9750-53ba46a6d2f1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:43.114617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:43.133635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:43.142454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:43.143106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:43.143414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:43.143663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:43.145764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ec352-51b2-4ce6-a32a-fdbd73bbdc9c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:43.150417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:43.151898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:57:43.163423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:57:43.164350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:57:43.165257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 552/1102] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 18:57:43.170152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:43.183589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:43.184212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:43.184483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:43.184755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:43.185850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] RBAC: Authorizing `identity:get_project(project_id=4c058a9e00ce4e64bda849628bbbc548)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:43.187297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:43.188904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 551/1103] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:57:43 2022] GET /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:43.195315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:43.273579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874b06-6dab-4830-a7ac-2c73f851b374 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:43.296476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef220c6-eddd-4694-8c8a-e4a05556c855 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:43.307533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:43.308137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:43.308415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:43.308687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:43.309375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:43.309682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:43.311156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:43.322893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 553/1104] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:43.328949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:43.330714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:43.331309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:43.331579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:43.331840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:43.332548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=4c058a9e00ce4e64bda849628bbbc548, user.id=441537905d6b4e22a018cbdfe8273679) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:43.332843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:43.334287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf8097-202c-40d7-a69b-7c357fe089fb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:43.340273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 552/1105] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:43 2022] GET /identity/v3/role_assignments?user.id=441537905d6b4e22a018cbdfe8273679&scope.project.id=4c058a9e00ce4e64bda849628bbbc548&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:43.359544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:43.445541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95656e9e-f9f6-407d-bc45-a3b423e1fe92 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:43.481005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:43.481616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:43.481891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:43.482161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:43.482937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] RBAC: Adding query filter params (role.id=3d46336886044db9a1c98315ee62695c, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=a500e08c87dc4e45899d7408e7b906cc) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:43.483246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:43.484745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b178d8f-e0b2-4467-893e-aca913d646f8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:43.491888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 554/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:43 2022] GET /identity/v3/role_assignments?user.id=a500e08c87dc4e45899d7408e7b906cc&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=3d46336886044db9a1c98315ee62695c => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:57:45.549098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa145b-8a28-48d0-b4e7-848d095be9d9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:45.549644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa145b-8a28-48d0-b4e7-848d095be9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:45.549644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa145b-8a28-48d0-b4e7-848d095be9d9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:45.550911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 553/1107] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:45.556095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:45.556489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:45.556640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:45.574378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:45.575582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:45.640515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 555/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 18:57:45.909299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa145b-8a28-48d0-b4e7-848d095be9d9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:45.930346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:46.008249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa145b-8a28-48d0-b4e7-848d095be9d9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:46.041743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b00773-cb81-47e6-a37f-548f861fe55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:46.042364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b00773-cb81-47e6-a37f-548f861fe55a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:46.042678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b00773-cb81-47e6-a37f-548f861fe55a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:46.042957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b00773-cb81-47e6-a37f-548f861fe55a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:46.044065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b00773-cb81-47e6-a37f-548f861fe55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:46.045526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b00773-cb81-47e6-a37f-548f861fe55a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:46.047112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 554/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:46.053353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:46.074501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:46.156059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb82a1-aef9-4819-88af-ba6e355b5b04 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:46.189661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec8472-b243-409f-a170-5006d2d88685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:46.190193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8472-b243-409f-a170-5006d2d88685 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:46.190473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8472-b243-409f-a170-5006d2d88685 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:46.190773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8472-b243-409f-a170-5006d2d88685 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:46.191426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec8472-b243-409f-a170-5006d2d88685 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:46.192867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec8472-b243-409f-a170-5006d2d88685 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:46.406267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 556/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:57:46 2022] POST /identity/v3/users => generated 286 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:57:48.555585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24806ea-83b9-4292-ab01-3eb66fadea13 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:48.556114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24806ea-83b9-4292-ab01-3eb66fadea13 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:48.556114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24806ea-83b9-4292-ab01-3eb66fadea13 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:48.557411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 555/1111] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:48.562493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:48.562898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:48.563064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:48.580593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:48.581749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:48.646966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 557/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:48.910163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24806ea-83b9-4292-ab01-3eb66fadea13 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:48.995346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:49.007971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24806ea-83b9-4292-ab01-3eb66fadea13 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:49.041855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:49.042471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:49.042780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:49.043057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] PATH_INFO: `/v3/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:49.044473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] RBAC: Authorizing `identity:get_role(role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:49.045860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:49.047192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 556/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:48 2022] GET /identity/v3/roles/b6752aeca4994d6292f5216c4d87e805 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:49.053412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:49.061473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:49.151642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2c9e5-3d7f-4a71-af03-491019ab3fd6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:49.185568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:49.186131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:49.186410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:49.186714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] PATH_INFO: `/v3/users/8618ab6bd6b545958a0c4f9513dc3201` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:49.200428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] RBAC: Authorizing `identity:get_user(user_id=8618ab6bd6b545958a0c4f9513dc3201)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:49.201861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:49.203478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 558/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:49 2022] GET /identity/v3/users/8618ab6bd6b545958a0c4f9513dc3201 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:49.209308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:49.305285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48783964-1cf7-4f91-8753-a470ed5872b5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:49.337720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:49.338308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:49.338603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:49.338875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:49.362768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:49.404058 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:49.405475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:49.501637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 557/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:57:49.507763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:49.568112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:49.604820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f009ef-b0b3-4588-8e6d-81a2c7e723e1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:49.638140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f09f-5365-45c0-b02a-b43040aad200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:49.638775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f09f-5365-45c0-b02a-b43040aad200 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:49.639056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f09f-5365-45c0-b02a-b43040aad200 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:49.639327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f09f-5365-45c0-b02a-b43040aad200 None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:49.640479 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5f09f-5365-45c0-b02a-b43040aad200 None admin] RBAC: Authorizing `identity:get_project(project_id=4c058a9e00ce4e64bda849628bbbc548)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:49.641896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5f09f-5365-45c0-b02a-b43040aad200 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:49.643517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 559/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:57:49 2022] GET /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:49.649554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:49.704600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:49.748064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8058aa0-4fd0-49e4-b430-466ed22542f5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:49.783588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:49.784114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:49.784389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:49.784660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:49.785379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] RBAC: Adding query filter params (role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=4c058a9e00ce4e64bda849628bbbc548, user.id=8618ab6bd6b545958a0c4f9513dc3201) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:49.785683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:49.787762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8444d-388b-4ce4-8cec-8e8cd7596b84 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:49.793032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 558/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:49 2022] GET /identity/v3/role_assignments?user.id=8618ab6bd6b545958a0c4f9513dc3201&scope.project.id=4c058a9e00ce4e64bda849628bbbc548&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:51.869823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2138-fcad-4c10-ab24-a9951106629c None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:51.870378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2138-fcad-4c10-ab24-a9951106629c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:51.870378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2138-fcad-4c10-ab24-a9951106629c None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:51.871718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 560/1118] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:51.876765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:51.877178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:51.877315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:51.895603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:51.896779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:51.960992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 559/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:52.235736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2138-fcad-4c10-ab24-a9951106629c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:52.334310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2138-fcad-4c10-ab24-a9951106629c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:52.362903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:52.367652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:52.368268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:52.368545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:52.368816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] PATH_INFO: `/v3/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:52.370194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] RBAC: Authorizing `identity:get_role(role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:52.371583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:52.372885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 561/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:52 2022] GET /identity/v3/roles/b6752aeca4994d6292f5216c4d87e805 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:52.378688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:52.427485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:52.475798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7ef6f-272c-44ae-a5ad-d6b599344b4d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:52.508216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:52.508776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:52.509052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:52.509324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] PATH_INFO: `/v3/users/8618ab6bd6b545958a0c4f9513dc3201` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:52.510419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] RBAC: Authorizing `identity:get_user(user_id=8618ab6bd6b545958a0c4f9513dc3201)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:52.511858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:52.513407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 560/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:52 2022] GET /identity/v3/users/8618ab6bd6b545958a0c4f9513dc3201 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:52.519076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:52.615967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5434ebe-4098-4bbd-9f72-b525a93ff6c0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:52.649161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d2c7e-b540-4a73-b170-760c50066155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:52.649783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d2c7e-b540-4a73-b170-760c50066155 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:52.650058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d2c7e-b540-4a73-b170-760c50066155 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:52.650328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d2c7e-b540-4a73-b170-760c50066155 None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:52.651458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d2c7e-b540-4a73-b170-760c50066155 None admin] RBAC: Authorizing `identity:get_project(project_id=4c058a9e00ce4e64bda849628bbbc548)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:52.652876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d2c7e-b540-4a73-b170-760c50066155 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:52.654475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 562/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:57:52 2022] GET /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:52.660625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:52.755714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530384d-cdeb-4434-9db0-878445e9edbb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:52.772957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:52.789768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fddf27-bbc2-4269-b0be-0dbd65551b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:52.790417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fddf27-bbc2-4269-b0be-0dbd65551b8d None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:52.790734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fddf27-bbc2-4269-b0be-0dbd65551b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:52.791011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fddf27-bbc2-4269-b0be-0dbd65551b8d None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548/users/8618ab6bd6b545958a0c4f9513dc3201/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:52.793156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fddf27-bbc2-4269-b0be-0dbd65551b8d None admin] RBAC: Authorizing `identity:create_grant(project_id=4c058a9e00ce4e64bda849628bbbc548, user_id=8618ab6bd6b545958a0c4f9513dc3201, role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:52.794592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fddf27-bbc2-4269-b0be-0dbd65551b8d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:52.844646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 561/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:57:52 2022] PUT /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548/users/8618ab6bd6b545958a0c4f9513dc3201/roles/b6752aeca4994d6292f5216c4d87e805 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:57:54.951063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d923790-4a7a-4a73-8cef-d0d0137ccf5c None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:54.951603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d923790-4a7a-4a73-8cef-d0d0137ccf5c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:54.951603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d923790-4a7a-4a73-8cef-d0d0137ccf5c None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:54.952963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 563/1124] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:54.958840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566adc8-985e-4112-8d32-d1887235808d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:54.959236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566adc8-985e-4112-8d32-d1887235808d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:54.959389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566adc8-985e-4112-8d32-d1887235808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:54.977794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5566adc8-985e-4112-8d32-d1887235808d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:54.979007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5566adc8-985e-4112-8d32-d1887235808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:55.044405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 562/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:55.311580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d923790-4a7a-4a73-8cef-d0d0137ccf5c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:55.378015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:55.411600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d923790-4a7a-4a73-8cef-d0d0137ccf5c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:55.445759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:55.446369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:55.446669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:55.446944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] PATH_INFO: `/v3/roles/b6752aeca4994d6292f5216c4d87e805` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:55.448329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] RBAC: Authorizing `identity:get_role(role_id=b6752aeca4994d6292f5216c4d87e805)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:55.449708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:55.451051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 564/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:55 2022] GET /identity/v3/roles/b6752aeca4994d6292f5216c4d87e805 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:55.456816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566adc8-985e-4112-8d32-d1887235808d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:55.551408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566adc8-985e-4112-8d32-d1887235808d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:55.561768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:55.588528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:55.589150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:55.589426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:55.589698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] PATH_INFO: `/v3/users/8618ab6bd6b545958a0c4f9513dc3201` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:55.590834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] RBAC: Authorizing `identity:get_user(user_id=8618ab6bd6b545958a0c4f9513dc3201)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:55.592295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:55.594227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 563/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:57:55 2022] GET /identity/v3/users/8618ab6bd6b545958a0c4f9513dc3201 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:55.599745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:55.697707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead31c9-e317-4091-8cd6-75df37fb33b2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:55.730075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:55.730695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:55.730973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:55.731246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:55.734489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:55.796948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:55.798387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:55.894462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 565/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:57:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:57:55.900448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:55.998041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1f32e-0fce-4c7c-9f7c-7a789de85756 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:56.030797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e6f37-a8db-40ac-a071-951f027cadd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:56.031405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6f37-a8db-40ac-a071-951f027cadd4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:56.031923 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6f37-a8db-40ac-a071-951f027cadd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:56.032196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6f37-a8db-40ac-a071-951f027cadd4 None admin] PATH_INFO: `/v3/projects/4c058a9e00ce4e64bda849628bbbc548` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:56.033288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e6f37-a8db-40ac-a071-951f027cadd4 None admin] RBAC: Authorizing `identity:get_project(project_id=4c058a9e00ce4e64bda849628bbbc548)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:56.034731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e6f37-a8db-40ac-a071-951f027cadd4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:56.036865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 564/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:57:55 2022] GET /identity/v3/projects/4c058a9e00ce4e64bda849628bbbc548 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:56.042902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:56.077019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:56.141357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a1ab4-4ef2-489f-97a4-4fa5d9c00b02 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:56.175297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:56.175823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:56.176097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:56.176363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:56.177775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] RBAC: Adding query filter params (role.id=b6752aeca4994d6292f5216c4d87e805, scope.project.id=4c058a9e00ce4e64bda849628bbbc548, user.id=8618ab6bd6b545958a0c4f9513dc3201) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:57:56.178085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:56.179526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61332ae6-c31f-4c82-92ef-63b8969ef042 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:56.185092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 566/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:57:56 2022] GET /identity/v3/role_assignments?user.id=8618ab6bd6b545958a0c4f9513dc3201&scope.project.id=4c058a9e00ce4e64bda849628bbbc548&role.id=b6752aeca4994d6292f5216c4d87e805 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:58.324562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c45-8a7a-47b2-9f07-10b4d109ee93 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:58.325067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c45-8a7a-47b2-9f07-10b4d109ee93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:58.325067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c45-8a7a-47b2-9f07-10b4d109ee93 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:58.326394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 565/1131] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:57:58.331637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:58.332019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:58.332200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:58.350484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:57:58.351698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:57:58.416732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 567/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:57:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:57:58.683270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c45-8a7a-47b2-9f07-10b4d109ee93 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:58.728292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:58.781185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c45-8a7a-47b2-9f07-10b4d109ee93 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:58.814557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033eb96-0a5b-43d7-a64f-720d91c05763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:58.815189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033eb96-0a5b-43d7-a64f-720d91c05763 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:58.815467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033eb96-0a5b-43d7-a64f-720d91c05763 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:58.815740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033eb96-0a5b-43d7-a64f-720d91c05763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:58.816844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033eb96-0a5b-43d7-a64f-720d91c05763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:58.818251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033eb96-0a5b-43d7-a64f-720d91c05763 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:58.819823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 566/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:57:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:57:58.825594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:57:58.854389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:57:58.922682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1537-df2b-4bbf-a66f-ec64703c7f43 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:57:58.955629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eae2d-4ca3-493f-9e65-6fd619202c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:57:58.956160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eae2d-4ca3-493f-9e65-6fd619202c63 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:57:58.956437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eae2d-4ca3-493f-9e65-6fd619202c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:57:58.956709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eae2d-4ca3-493f-9e65-6fd619202c63 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:57:58.957345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eae2d-4ca3-493f-9e65-6fd619202c63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:57:58.958821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eae2d-4ca3-493f-9e65-6fd619202c63 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:57:59.057852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 568/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:57:58 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:58:01.262900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfc4aa-6f89-4222-8ce8-1af0f6a44bcd None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:01.263340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfc4aa-6f89-4222-8ce8-1af0f6a44bcd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:01.263426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfc4aa-6f89-4222-8ce8-1af0f6a44bcd None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:01.264759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 567/1135] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:01.269896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:01.270306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:01.270466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:01.289020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:01.290200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:01.354672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 569/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:01.635803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfc4aa-6f89-4222-8ce8-1af0f6a44bcd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:01.735729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfc4aa-6f89-4222-8ce8-1af0f6a44bcd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:01.748170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:01.767774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e8ea2-acf9-4a62-a5da-12306ba09def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:01.777198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e8ea2-acf9-4a62-a5da-12306ba09def None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:01.777198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e8ea2-acf9-4a62-a5da-12306ba09def None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:01.777198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e8ea2-acf9-4a62-a5da-12306ba09def None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:01.777198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e8ea2-acf9-4a62-a5da-12306ba09def None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:01.777198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e8ea2-acf9-4a62-a5da-12306ba09def None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:01.777198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 568/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:58:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:01.779276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:01.793860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:01.884891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c55c08-c54a-4f82-8471-73f31fc6df90 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:01.918105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da63b-0286-4541-8497-58059096a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:01.918667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da63b-0286-4541-8497-58059096a14a None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:01.918950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da63b-0286-4541-8497-58059096a14a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:01.919223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da63b-0286-4541-8497-58059096a14a None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:01.919878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059da63b-0286-4541-8497-58059096a14a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:01.921312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059da63b-0286-4541-8497-58059096a14a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:01.981525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 570/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:58:01 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:58:04.152455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df440c02-2078-435c-a324-8f85ef1fc89f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:04.153051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df440c02-2078-435c-a324-8f85ef1fc89f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:04.153051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df440c02-2078-435c-a324-8f85ef1fc89f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:04.154306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 569/1139] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:04.159741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:04.160163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:04.160798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:04.179159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:04.180339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:04.240408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:04.245770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 571/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:04.522451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df440c02-2078-435c-a324-8f85ef1fc89f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:04.620190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df440c02-2078-435c-a324-8f85ef1fc89f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:04.653312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:04.653954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:04.654229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:04.654502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:04.665184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:58:04.669872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:04.671049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:58:04.671954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 570/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:58:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:04.676890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:04.773724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099ac71-6397-4aa0-b1b3-ef91fdb1b2f6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:04.806436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:04.807107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:04.807379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:04.807644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:04.808293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:04.808588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:04.809937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:04.818189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 572/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:04.824083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:04.867009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:04.922774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458ba17-4a76-457f-ba8e-540922364cf7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:04.955799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:04.956409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:04.956684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:04.956957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] PATH_INFO: `/v3/users/e549906f6d724bdcb5b177def1794519` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:04.971695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] RBAC: Authorizing `identity:get_user(user_id=e549906f6d724bdcb5b177def1794519)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:04.973179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:04.974784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 571/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:58:04 2022] GET /identity/v3/users/e549906f6d724bdcb5b177def1794519 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:04.980630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:05.038471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:05.078714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96994899-1b1b-40eb-9565-a2b435d8f809 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:05.111256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:05.111851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:05.112130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:05.112403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:05.181665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:05.183145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:05.239264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:05.279880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 573/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:58:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:58:05.286024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:05.377782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:05.382029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f84cdf-753b-496c-b05b-597d9a56db99 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:05.414938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f69d4-75c1-4b61-817e-feef77b8da34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:05.415551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f69d4-75c1-4b61-817e-feef77b8da34 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:05.415826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f69d4-75c1-4b61-817e-feef77b8da34 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:05.416098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f69d4-75c1-4b61-817e-feef77b8da34 None admin] PATH_INFO: `/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:05.417190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f69d4-75c1-4b61-817e-feef77b8da34 None admin] RBAC: Authorizing `identity:get_project(project_id=c31863a6931f4c82bd4df8b80ba9c6c1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:05.418619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f69d4-75c1-4b61-817e-feef77b8da34 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:05.420221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 572/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:58:05 2022] GET /identity/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:05.426981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:05.522724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2672f9-ab25-446e-be3e-4fda0e26e9ff None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:05.556906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:05.557499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:05.557776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:05.558048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:05.558790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=c31863a6931f4c82bd4df8b80ba9c6c1, user.id=e549906f6d724bdcb5b177def1794519) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:05.559095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:05.560505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883afb0-fb83-4589-822b-ac450e656a39 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:05.565794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 574/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:58:05 2022] GET /identity/v3/role_assignments?user.id=e549906f6d724bdcb5b177def1794519&scope.project.id=c31863a6931f4c82bd4df8b80ba9c6c1&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:07.674881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903323f4-1c31-4e69-a54d-c9791d88f844 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:07.675443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903323f4-1c31-4e69-a54d-c9791d88f844 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:07.675443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903323f4-1c31-4e69-a54d-c9791d88f844 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:07.676751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 573/1147] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:07.681847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:07.682266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:07.682385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:07.700195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:07.701372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:07.718530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:07.766671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 575/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:08.034406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903323f4-1c31-4e69-a54d-c9791d88f844 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:08.131655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903323f4-1c31-4e69-a54d-c9791d88f844 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:08.164087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:08.164686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:08.164957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:08.165223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:08.176927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:58:08.181587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:08.182694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:58:08.183547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 574/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:58:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:08.188537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:08.285638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccdbd9-d847-4a14-86dd-7b3e3809b618 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:08.319412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:08.319992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:08.320266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:08.320532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:08.321181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:08.321478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:08.322855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:08.330575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 576/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:08.336248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:08.363472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:08.432506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be46525-6f52-49c0-a7d5-87c6d117b837 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:08.464731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:08.465330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:08.465602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:08.465869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] PATH_INFO: `/v3/users/e549906f6d724bdcb5b177def1794519` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:08.467361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] RBAC: Authorizing `identity:get_user(user_id=e549906f6d724bdcb5b177def1794519)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:08.468939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:08.470486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 575/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:58:08 2022] GET /identity/v3/users/e549906f6d724bdcb5b177def1794519 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:08.476247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:08.494968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:08.574903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4a0a0-dadb-4608-bf7d-381c5a9e9f27 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:08.607285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc36ed-f0f9-4588-9cf0-ece3042c68fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:08.607900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc36ed-f0f9-4588-9cf0-ece3042c68fc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:08.608179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc36ed-f0f9-4588-9cf0-ece3042c68fc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:08.608453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc36ed-f0f9-4588-9cf0-ece3042c68fc None admin] PATH_INFO: `/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:08.609559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc36ed-f0f9-4588-9cf0-ece3042c68fc None admin] RBAC: Authorizing `identity:get_project(project_id=c31863a6931f4c82bd4df8b80ba9c6c1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:08.611023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc36ed-f0f9-4588-9cf0-ece3042c68fc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:08.612636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 577/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:58:08 2022] GET /identity/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:08.619223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:08.709665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:08.716060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fdb8-79e6-48b7-aa66-b0d6710f2625 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:08.749759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3c0c5-fc47-4000-9ff0-0432fb18d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:08.750397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3c0c5-fc47-4000-9ff0-0432fb18d1ce None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:08.750703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3c0c5-fc47-4000-9ff0-0432fb18d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:08.750978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3c0c5-fc47-4000-9ff0-0432fb18d1ce None admin] PATH_INFO: `/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1/users/e549906f6d724bdcb5b177def1794519/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:08.752851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3c0c5-fc47-4000-9ff0-0432fb18d1ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c31863a6931f4c82bd4df8b80ba9c6c1, user_id=e549906f6d724bdcb5b177def1794519, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:08.754255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3c0c5-fc47-4000-9ff0-0432fb18d1ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:08.838135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 576/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:58:08 2022] PUT /identity/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1/users/e549906f6d724bdcb5b177def1794519/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:58:10.956484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edfec1-a8ea-4183-b2a0-868ad23b712c None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:10.957021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edfec1-a8ea-4183-b2a0-868ad23b712c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:10.957021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edfec1-a8ea-4183-b2a0-868ad23b712c None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:10.958362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 578/1154] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 18:58:10.963578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:10.964021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:10.964116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:10.983059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:10.984272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:11.050492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 577/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:11.329881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edfec1-a8ea-4183-b2a0-868ad23b712c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:11.411543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:11.427737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edfec1-a8ea-4183-b2a0-868ad23b712c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:11.460288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:11.460891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:11.461162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:11.461426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:11.470989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:58:11.475717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:11.476783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:58:11.477775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 579/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:11.482669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:11.565698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:11.580927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf35d6-85f5-4b26-9c36-fce9404369ea None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:11.615495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:11.616102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:11.616382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:11.616652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:11.617312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:11.617612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:11.619000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:11.626841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 578/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:11.632930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:11.730323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fef1b-afb2-4920-a5f3-b29791d411a2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:11.763124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:11.763726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:11.764000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:11.764271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] PATH_INFO: `/v3/users/e549906f6d724bdcb5b177def1794519` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:11.765376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] RBAC: Authorizing `identity:get_user(user_id=e549906f6d724bdcb5b177def1794519)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:11.766819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:11.768363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 580/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:58:11 2022] GET /identity/v3/users/e549906f6d724bdcb5b177def1794519 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:11.774204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:11.874321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c771a1-9758-47c1-a563-6e230b6b4684 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:11.907222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:11.907815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:11.908092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:11.908363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:11.923500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:11.974043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:11.975503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:12.069392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 579/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:58:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:58:12.075365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:12.090153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:12.173420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24b86-9155-4093-abb3-faa47d8e1e37 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:12.207053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162d338-79fe-4fc8-a143-402e63541dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:12.207662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162d338-79fe-4fc8-a143-402e63541dec None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:12.207939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162d338-79fe-4fc8-a143-402e63541dec None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:12.208211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162d338-79fe-4fc8-a143-402e63541dec None admin] PATH_INFO: `/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:12.209322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162d338-79fe-4fc8-a143-402e63541dec None admin] RBAC: Authorizing `identity:get_project(project_id=c31863a6931f4c82bd4df8b80ba9c6c1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:12.210788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162d338-79fe-4fc8-a143-402e63541dec None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:12.212427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 581/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:58:12 2022] GET /identity/v3/projects/c31863a6931f4c82bd4df8b80ba9c6c1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:12.218466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:12.263880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:12.316733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b631795-455f-4bde-9153-f0cd90c67e46 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:12.349682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:12.350272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:12.350584 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:12.350851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:12.351568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=c31863a6931f4c82bd4df8b80ba9c6c1, user.id=e549906f6d724bdcb5b177def1794519) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:12.351876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:12.353283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6d2cb-3fa8-448f-9c96-2849b69e0edf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:12.358879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 580/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:58:12 2022] GET /identity/v3/role_assignments?user.id=e549906f6d724bdcb5b177def1794519&scope.project.id=c31863a6931f4c82bd4df8b80ba9c6c1&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:14.521834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a14cf-3ba4-4b2f-8647-095edbb1712a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:14.522358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a14cf-3ba4-4b2f-8647-095edbb1712a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:14.522358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a14cf-3ba4-4b2f-8647-095edbb1712a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:14.523664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 582/1162] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:14.528791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:14.529197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:14.529322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:14.547827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:14.548993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:14.613740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 581/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:14.883799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a14cf-3ba4-4b2f-8647-095edbb1712a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:14.964314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:14.983922 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a14cf-3ba4-4b2f-8647-095edbb1712a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:15.017152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:15.017749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:15.018027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:15.018301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:15.084536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:15.085973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:15.163694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:15.185413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 583/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:58:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:58:15.191495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:15.234839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:15.291421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e0595-9848-4a0a-8a98-c838ed839679 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:15.324226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:15.324842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:15.325122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:15.325395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:15.332447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:15.333866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d5771be1-29b7-4372-8b36-723b3713f24c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:15.344924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:15.346131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:58:15.347360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 582/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 18:58:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:15.352020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:15.447937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be3633-7303-476b-8d84-65a9d697dc8b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:15.480924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:15.481518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:15.481797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:15.482070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:15.482731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:15.483037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:15.484395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d09921-8326-481d-a70b-a878ed473272 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:15.491302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 584/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Sep 7 18:58:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:17.619015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f8a2-7d2b-44a8-8b2f-52a387146a91 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:17.619586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f8a2-7d2b-44a8-8b2f-52a387146a91 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:17.619586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f8a2-7d2b-44a8-8b2f-52a387146a91 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:17.620864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 583/1167] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:17.626047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de7e7-98fb-484b-a852-12262b45d8dd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:17.626444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de7e7-98fb-484b-a852-12262b45d8dd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:17.626627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de7e7-98fb-484b-a852-12262b45d8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:17.645487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6c5de7e7-98fb-484b-a852-12262b45d8dd None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:17.646702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6c5de7e7-98fb-484b-a852-12262b45d8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:17.684571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:17.711201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 585/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:17.975394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f8a2-7d2b-44a8-8b2f-52a387146a91 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:18.071074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:18.074785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f8a2-7d2b-44a8-8b2f-52a387146a91 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:18.106826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548346f9-40ab-418e-bc5f-60f312415aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:18.107418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548346f9-40ab-418e-bc5f-60f312415aef None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:18.107723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548346f9-40ab-418e-bc5f-60f312415aef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:18.108023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548346f9-40ab-418e-bc5f-60f312415aef None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:18.108698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548346f9-40ab-418e-bc5f-60f312415aef None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:18.110046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548346f9-40ab-418e-bc5f-60f312415aef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:18.170096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 584/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Wed Sep 7 18:58:17 2022] POST /identity/v3/domains => generated 267 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:58:20.337241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1d578-dd1c-4cdc-8f9d-36b03b6e88fd None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:20.337796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1d578-dd1c-4cdc-8f9d-36b03b6e88fd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:20.337796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1d578-dd1c-4cdc-8f9d-36b03b6e88fd None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:20.339149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 586/1170] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:20.344462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:20.344843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:20.345021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:20.363358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:20.364546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:20.428880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 585/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:20.699044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1d578-dd1c-4cdc-8f9d-36b03b6e88fd None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:20.797510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1d578-dd1c-4cdc-8f9d-36b03b6e88fd None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:20.830269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58464bb7-51fd-479a-8fce-039b9cf9c0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:20.830925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58464bb7-51fd-479a-8fce-039b9cf9c0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:20.831199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58464bb7-51fd-479a-8fce-039b9cf9c0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:20.831468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58464bb7-51fd-479a-8fce-039b9cf9c0a2 None admin] PATH_INFO: `/v3/domains/e573afb931934dbaa5657da72c13febe` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:20.841462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58464bb7-51fd-479a-8fce-039b9cf9c0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e573afb931934dbaa5657da72c13febe)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:20.842927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58464bb7-51fd-479a-8fce-039b9cf9c0a2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:20.844495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 587/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Wed Sep 7 18:58:20 2022] GET /identity/v3/domains/e573afb931934dbaa5657da72c13febe => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:20.850719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:20.937431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:20.946743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78106c-dae5-4f80-9139-ca302b483a2e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:20.996181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afb09e-faa8-4195-b3c7-fe237dd685d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:20.996181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afb09e-faa8-4195-b3c7-fe237dd685d6 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:20.996181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afb09e-faa8-4195-b3c7-fe237dd685d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:20.996181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afb09e-faa8-4195-b3c7-fe237dd685d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:20.996635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afb09e-faa8-4195-b3c7-fe237dd685d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:20.996635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afb09e-faa8-4195-b3c7-fe237dd685d6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:21.057394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 586/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 18:58:20 2022] POST /identity/v3/projects => generated 361 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:58:23.258895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a7fa1-5d11-4f4a-bceb-fc427f9ed34a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:23.259358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a7fa1-5d11-4f4a-bceb-fc427f9ed34a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:23.259469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a7fa1-5d11-4f4a-bceb-fc427f9ed34a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:23.260760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 588/1174] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:23.266084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:23.266505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:23.266670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:23.284741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:23.285921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:23.351500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 587/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:23.623839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a7fa1-5d11-4f4a-bceb-fc427f9ed34a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:23.659051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:23.723653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a7fa1-5d11-4f4a-bceb-fc427f9ed34a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:23.755995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f979192-07a3-4332-9a28-b9a29f5a63d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:23.756609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f979192-07a3-4332-9a28-b9a29f5a63d1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:23.756889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f979192-07a3-4332-9a28-b9a29f5a63d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:23.757162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f979192-07a3-4332-9a28-b9a29f5a63d1 None admin] PATH_INFO: `/v3/domains/e573afb931934dbaa5657da72c13febe` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:23.758270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f979192-07a3-4332-9a28-b9a29f5a63d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e573afb931934dbaa5657da72c13febe)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:23.759737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f979192-07a3-4332-9a28-b9a29f5a63d1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:23.761294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 589/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Wed Sep 7 18:58:23 2022] GET /identity/v3/domains/e573afb931934dbaa5657da72c13febe => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:23.767256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:23.833304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:23.860855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f353965-aa70-48cd-8fa6-a96b80411d8b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:23.894695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5ac16-9233-46d4-a7c5-e9a333860346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:23.895227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5ac16-9233-46d4-a7c5-e9a333860346 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:23.895504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5ac16-9233-46d4-a7c5-e9a333860346 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:23.895775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5ac16-9233-46d4-a7c5-e9a333860346 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:23.896410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5ac16-9233-46d4-a7c5-e9a333860346 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:23.897820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5ac16-9233-46d4-a7c5-e9a333860346 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:24.008910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 588/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:58:23 2022] POST /identity/v3/users => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 18:58:26.197245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38a74c-2ad4-42a0-a64b-86d6c21de42b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:26.197821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38a74c-2ad4-42a0-a64b-86d6c21de42b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:26.197821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38a74c-2ad4-42a0-a64b-86d6c21de42b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:26.199137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 590/1178] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:26.204436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:26.204847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:26.204990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:26.222952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:26.224148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:26.290144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 589/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:26.563475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38a74c-2ad4-42a0-a64b-86d6c21de42b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:26.664694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38a74c-2ad4-42a0-a64b-86d6c21de42b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:26.666165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:26.698058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:26.698750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:26.699032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:26.699305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:26.709668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:58:26.714410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:26.715618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:58:26.716871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 591/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:58:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:26.722151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:26.777909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:26.819185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55187dd-b4b8-415d-9ea7-6cd684002c6b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:26.852350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:26.852938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:26.853215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:26.853509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:26.854163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:26.854461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:26.855839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:26.864176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 590/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:58:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:26.870125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:26.967894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5bdc8-0fcd-43c7-9dbc-abb13ae81481 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:27.000388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:27.000996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:27.001279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:27.001552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] PATH_INFO: `/v3/users/bb124b882fab4882941991e57a523bc2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:27.014236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] RBAC: Authorizing `identity:get_user(user_id=bb124b882fab4882941991e57a523bc2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:27.015708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:27.017287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 592/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:58:26 2022] GET /identity/v3/users/bb124b882fab4882941991e57a523bc2 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:27.023487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:27.121875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8932d-33e8-454d-a282-a73eb9c0c8f2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:27.141296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:27.154725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:27.155582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:27.155858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:27.156152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:27.221475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:27.222938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:27.318202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 591/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:58:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:58:27.324214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:27.345512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:27.420759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de616ee8-90ef-4443-8c2f-160b9007e62a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:27.453110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb571cb-c187-4cd9-80a3-d83957b8af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:27.453717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb571cb-c187-4cd9-80a3-d83957b8af2f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:27.453990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb571cb-c187-4cd9-80a3-d83957b8af2f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:27.454256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb571cb-c187-4cd9-80a3-d83957b8af2f None admin] PATH_INFO: `/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:27.455413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb571cb-c187-4cd9-80a3-d83957b8af2f None admin] RBAC: Authorizing `identity:get_project(project_id=ee4de635777c43679cd8a3a9eeee2d5b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:27.456822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb571cb-c187-4cd9-80a3-d83957b8af2f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:27.458409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 593/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:58:27 2022] GET /identity/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:27.464559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:27.485531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:27.562217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b370e-99d8-4081-a50c-0706da88a778 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:27.594311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:27.594926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:27.595207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:27.595480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:27.596724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=ee4de635777c43679cd8a3a9eeee2d5b, user.id=bb124b882fab4882941991e57a523bc2) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:27.597032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:27.598439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c905f-0629-499e-955b-8b9de7ec957d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:27.603609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 592/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:58:27 2022] GET /identity/v3/role_assignments?user.id=bb124b882fab4882941991e57a523bc2&scope.project.id=ee4de635777c43679cd8a3a9eeee2d5b&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:29.713577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cb2f0-be9d-4ca5-87cc-86c72242e758 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:29.714133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cb2f0-be9d-4ca5-87cc-86c72242e758 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:29.714133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cb2f0-be9d-4ca5-87cc-86c72242e758 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:29.715565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 594/1186] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:29.721279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:29.721709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:29.721804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:29.740430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:29.741616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:29.806156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 593/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:30.076454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cb2f0-be9d-4ca5-87cc-86c72242e758 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:30.163872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:30.176257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cb2f0-be9d-4ca5-87cc-86c72242e758 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:30.209346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:30.209964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:30.210240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:30.210513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:30.221219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:58:30.225941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:30.227041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:58:30.228522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 595/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:58:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:30.233105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:30.248068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:30.332337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e919-8862-45cc-8c47-dc895a334ba8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:30.364566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:30.365166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:30.365446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:30.365718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:30.366383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:30.366712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:30.368077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:30.375844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 594/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:58:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:30.382234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:30.484089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad0469-c45b-41bf-a388-77841ad18f06 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:30.516667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:30.517875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:30.518162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:30.518433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] PATH_INFO: `/v3/users/bb124b882fab4882941991e57a523bc2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:30.519592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] RBAC: Authorizing `identity:get_user(user_id=bb124b882fab4882941991e57a523bc2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:30.521025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:30.522610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 596/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:58:30 2022] GET /identity/v3/users/bb124b882fab4882941991e57a523bc2 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:30.528434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:30.606439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:30.629322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f260c-f819-4b0b-82cb-2b565c219445 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:30.662441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce4dd-693e-428f-a274-096b199ad8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:30.663090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce4dd-693e-428f-a274-096b199ad8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:30.663366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce4dd-693e-428f-a274-096b199ad8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:30.663638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce4dd-693e-428f-a274-096b199ad8d4 None admin] PATH_INFO: `/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:30.664739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce4dd-693e-428f-a274-096b199ad8d4 None admin] RBAC: Authorizing `identity:get_project(project_id=ee4de635777c43679cd8a3a9eeee2d5b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:30.666161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce4dd-693e-428f-a274-096b199ad8d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:30.667815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 595/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:58:30 2022] GET /identity/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:30.674246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:30.693520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:30.772442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0af031-2d9d-4617-8351-6e88622a60a4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:30.804666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4850c5-bf82-46f5-8080-186d23934c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:30.805301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4850c5-bf82-46f5-8080-186d23934c8e None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:30.805577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4850c5-bf82-46f5-8080-186d23934c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:30.805849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4850c5-bf82-46f5-8080-186d23934c8e None admin] PATH_INFO: `/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b/users/bb124b882fab4882941991e57a523bc2/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:30.807738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4850c5-bf82-46f5-8080-186d23934c8e None admin] RBAC: Authorizing `identity:create_grant(project_id=ee4de635777c43679cd8a3a9eeee2d5b, user_id=bb124b882fab4882941991e57a523bc2, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:30.809235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4850c5-bf82-46f5-8080-186d23934c8e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:30.902880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 597/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:58:30 2022] PUT /identity/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b/users/bb124b882fab4882941991e57a523bc2/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:58:33.009942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e458a-db7b-45e5-92f5-b73f05435505 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:33.010503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e458a-db7b-45e5-92f5-b73f05435505 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:33.010503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e458a-db7b-45e5-92f5-b73f05435505 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:33.011824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 596/1193] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:58:33.016973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:33.017913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:33.018055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:33.036960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:58:33.038133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:58:33.103747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 598/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:58:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:58:33.373796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e458a-db7b-45e5-92f5-b73f05435505 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:33.471995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e458a-db7b-45e5-92f5-b73f05435505 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:33.503613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:33.504215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:33.504489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:33.504753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:33.514821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:58:33.520087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:58:33.521198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:58:33.522425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 597/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 18:58:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:58:33.527038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:33.535246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:33.623299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553509e7-679d-46ed-b3b7-5a9deecacc18 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:33.656633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:33.657171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:33.657449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:33.657758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:33.658410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:33.658738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:33.660100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:33.667848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 599/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 18:58:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:33.674317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:33.678024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:33.774725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158421f-597a-4a6e-ae00-0a32bdfc2e99 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:33.807042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:33.807648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:33.807924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:33.808195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] PATH_INFO: `/v3/users/bb124b882fab4882941991e57a523bc2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:33.809289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] RBAC: Authorizing `identity:get_user(user_id=bb124b882fab4882941991e57a523bc2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:33.810728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:33.812270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 598/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Sep 7 18:58:33 2022] GET /identity/v3/users/bb124b882fab4882941991e57a523bc2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:33.818775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:33.883661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:33.916049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548d8f8-37cb-476c-a098-85d310e13fe2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:33.949760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:33.950349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:33.950661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:33.950933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:34.015282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:34.016713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:34.078667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:34.111492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 600/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:58:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:58:34.117514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:34.185158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:34.212964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549a25-d522-4855-83d7-123a57f528bb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:34.246311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b95441-363c-4734-b310-707fa6fc776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:34.246962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b95441-363c-4734-b310-707fa6fc776b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:34.247240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b95441-363c-4734-b310-707fa6fc776b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:34.247510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b95441-363c-4734-b310-707fa6fc776b None admin] PATH_INFO: `/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:34.248604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b95441-363c-4734-b310-707fa6fc776b None admin] RBAC: Authorizing `identity:get_project(project_id=ee4de635777c43679cd8a3a9eeee2d5b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:34.250010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b95441-363c-4734-b310-707fa6fc776b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:34.251692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 599/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Sep 7 18:58:34 2022] GET /identity/v3/projects/ee4de635777c43679cd8a3a9eeee2d5b => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:58:34.257761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:58:34.353408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8d4f1-48c5-4670-b656-121991366bb6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:58:34.387810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:58:34.388396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:58:34.388670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:58:34.388942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:58:34.389658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=ee4de635777c43679cd8a3a9eeee2d5b, user.id=bb124b882fab4882941991e57a523bc2) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:58:34.389962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:58:34.391398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946a6a-da73-454d-a0a1-c34d75702e25 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:58:34.397492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 601/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:58:34 2022] GET /identity/v3/role_assignments?user.id=bb124b882fab4882941991e57a523bc2&scope.project.id=ee4de635777c43679cd8a3a9eeee2d5b&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 18:58:44.242140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=30 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:58:44.394984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=84 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:45.005811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b82de2-2b9f-45d0-9030-24e3d49d9b56 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:45.006402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b82de2-2b9f-45d0-9030-24e3d49d9b56 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:45.006402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b82de2-2b9f-45d0-9030-24e3d49d9b56 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:45.007721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 600/1201] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:59:45.012940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:45.013374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:45.013495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:45.031767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:59:45.033453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:59:45.099654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 602/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:59:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:59:45.375444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b82de2-2b9f-45d0-9030-24e3d49d9b56 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:45.480188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b82de2-2b9f-45d0-9030-24e3d49d9b56 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:45.512370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:45.514429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:45.514429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:45.514429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:45.524120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:45.525526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:45.539856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:45.540953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:45.542415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:45.542415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:45.542415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:59:45.542415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 601/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:45.547295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:45.645813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e4edc-a053-4fcb-8e23-e61d4ec7cf43 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:45.673805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:45.680635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:45.681243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:45.681537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:45.681811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:45.682454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:45.682789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:45.684800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818c061-bda7-4b89-8da7-a6be80e82eb9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:45.696744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 603/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:45.702745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:45.754906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:45.800253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0dd60-26a6-4253-8f21-263ff4c6c9dd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:45.834806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfb83b-1256-4790-96c9-0fbd067514ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:45.835405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfb83b-1256-4790-96c9-0fbd067514ba None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:45.835681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfb83b-1256-4790-96c9-0fbd067514ba None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:45.835951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfb83b-1256-4790-96c9-0fbd067514ba None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:45.836601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfb83b-1256-4790-96c9-0fbd067514ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:45.838040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfb83b-1256-4790-96c9-0fbd067514ba None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:45.916682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 602/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Sep 7 18:59:45 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 18:59:48.091874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe23ba-e72c-4abd-a20d-9889412ab0be None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:48.092514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe23ba-e72c-4abd-a20d-9889412ab0be None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:48.092514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe23ba-e72c-4abd-a20d-9889412ab0be None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:48.093958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 604/1206] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:59:48.099328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:48.100880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:48.100880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:48.118121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:59:48.121917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:59:48.189494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 603/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:59:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:59:48.462042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe23ba-e72c-4abd-a20d-9889412ab0be None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:48.564727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe23ba-e72c-4abd-a20d-9889412ab0be None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:48.598797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:48.599425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:48.599702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:48.599976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:48.611205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:59:48.616009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:48.617911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:48.619993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 605/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:59:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:48.625650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:48.665830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:48.724780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c7236-d9ca-4aa0-ba35-f6b0a886bccf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:48.758438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:48.759088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:48.759366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:48.759639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:48.760300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:48.760719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:48.762075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:48.769821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 604/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:59:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:48.776100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:48.826219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:48.875872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07648c3-b8c1-4423-bc8f-b86c0cb13c84 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:48.909669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:48.910341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:48.910866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:48.911319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:48.923820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:48.925283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:48.940080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:48.941229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:48.942365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:48.942365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:48.942365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:48.942365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 606/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:48.947269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:49.051048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:49.054066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22636a-970a-4537-9843-3d498800c3ae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:49.086911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:49.087512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:49.087791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:49.088062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:49.088706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:49.089066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:49.090411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:49.102416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 605/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:49.108279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:49.211248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ff86b-650d-4f3e-9a09-144ad1ac3265 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:49.212800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:49.245758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:49.246389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:49.246713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:49.246990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:49.255717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:49.261420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:59:49.269169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:49.270198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:49.271522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 607/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:49.276584 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:49.379966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0172f-d638-43ed-9bc1-91676ffac4a2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:49.413660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:49.414263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:49.414542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:49.414850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:49.415577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:49.415881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:49.417300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:49.434113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 606/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Sep 7 18:59:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:49.440652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:49.540802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45047af0-9583-4b32-9329-ed39a6215d6e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:49.573210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:49.573802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:49.574073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:49.574338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:49.603332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:49.641705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:49.643567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:49.741238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 608/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:59:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:59:49.747267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:49.767002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:49.845735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be52af4-fd7b-4a62-9036-4984c7491267 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:49.879069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:49.879690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:49.879969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:49.880240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:49.889859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:49.891341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:49.905691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:49.906972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:49.908177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:49.908177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:49.908177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:59:49.908177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 607/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:49.912462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:49.972435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:50.010797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b7f09-3198-47fe-82c3-24bb8b8d1cb0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:50.045327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:50.045931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:50.046210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:50.046486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:50.047822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:50.048132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:50.049591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:50.060892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 609/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:50.066803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:50.135279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:50.167125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff7ec1-57aa-45cb-9cd8-24fb56db1846 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:50.200739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:50.201354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:50.201632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:50.201901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:50.208450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:50.209886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:59:50.221467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:50.222607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:59:50.223915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 608/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:50.228529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:50.327046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7371-31ff-4079-b9ea-2216ea745c0b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:50.350957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:50.363110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:50.363714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:50.363986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:50.364253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:50.364934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:50.365235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:50.366691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b604-0b9c-4f5b-88a7-18bf44e529d7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:50.378854 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 610/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:50.384938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:50.483985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdf425-9b4f-4513-9a57-8fc9a44cfeb6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:50.512273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:50.518069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:50.518714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:50.519005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:50.519298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:50.520027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=c32b1292d0c6436ba4669dbfd1c73ea4) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:50.520334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:50.521783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cde5e-a89a-4047-85ad-139f7b89cd49 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:50.527333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 609/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:59:50 2022] GET /identity/v3/role_assignments?user.id=c32b1292d0c6436ba4669dbfd1c73ea4&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:52.652191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6317e-c243-4107-a5d9-f508872f78c6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:52.652809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6317e-c243-4107-a5d9-f508872f78c6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:52.652809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6317e-c243-4107-a5d9-f508872f78c6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:52.654139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 611/1220] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:59:52.659875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:52.660299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:52.660416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:52.679105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:59:52.680286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:59:52.746978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 610/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:59:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:59:53.021966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6317e-c243-4107-a5d9-f508872f78c6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:53.123585 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6317e-c243-4107-a5d9-f508872f78c6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:53.157825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:53.158443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:53.158754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:53.159029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:53.170534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:59:53.175431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:53.176517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:53.177419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 612/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:53.182535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:53.282020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932a56d-d7d1-4319-810d-ab1cd6b2beb0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:53.304939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:53.315130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:53.315726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:53.316002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:53.316270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:53.316919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:53.317214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:53.318564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:53.326132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 611/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:53.332070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:53.377302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:53.432258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfeaf-a84a-42f3-a51d-3626c8077af5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:53.465080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:53.465910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:53.466785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:53.467148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:53.477613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:53.479148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:53.493520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:53.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:53.495976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:53.495976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:53.495976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:53.495976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 613/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:53.500534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:53.599147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb417154-c553-46df-9d41-a51470d4279d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:53.631546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:53.632143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:53.632418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:53.632681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:53.633312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:53.633610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:53.634974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:53.646966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 612/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:53.653034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:53.745140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:53.752253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea7499-2701-4cf6-8ba8-af8a31320f3d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:53.786646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:53.787445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:53.787742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:53.788067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:53.795788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:53.797581 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:59:53.809129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:53.810275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:53.811471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 614/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:53.816908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:53.863591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:53.916141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7409ca-cd55-48a1-88ae-11fedf7c41b0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:53.949345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:53.949945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:53.950223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:53.950497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:53.951240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:53.951544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:53.952962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:53.970947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 613/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:53.976723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:54.076947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104e78-70de-4edd-8ee5-fc3457a2d0a2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:54.110768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:54.111377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:54.111656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:54.111929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:54.122508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:54.123933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:54.138279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:54.139391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:54.140292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:54.140292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:54.140292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:54.140292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 615/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:54.149713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:54.253036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330f6f9-d14b-4de1-ac57-265ba0f33260 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:54.255272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:54.286196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:54.286824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:54.287105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:54.287378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:54.288016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:54.288317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:54.289663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:54.301049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 614/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:54.306686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:54.317420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:54.405924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9d6b-37ad-4c78-9a75-809e31f4640b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:54.440150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:54.440767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:54.441043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:54.441317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:54.447827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:54.449257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:59:54.460276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:54.461346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:54.462419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 616/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:59:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:54.467318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:54.565986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a3f76-0c7f-44fa-b700-dd6178dcffaa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:54.598947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:54.599542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:54.599818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:54.600089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:54.600778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:54.601076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:54.602480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e47605-de66-40e1-8866-e077e085f133 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:54.614815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 615/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:59:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:54.620943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:54.687085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:54.720453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b3894-e763-48d0-9b5c-5f7b527b3e4c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:54.753208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f26b64-2a4e-40dc-b50e-a3076c971be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:54.753848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f26b64-2a4e-40dc-b50e-a3076c971be0 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:54.754120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f26b64-2a4e-40dc-b50e-a3076c971be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:54.754388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f26b64-2a4e-40dc-b50e-a3076c971be0 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/c32b1292d0c6436ba4669dbfd1c73ea4/roles/be6b18fac0ca4153b9530e8b1a33bf64` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:54.768615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f26b64-2a4e-40dc-b50e-a3076c971be0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=c32b1292d0c6436ba4669dbfd1c73ea4, role_id=be6b18fac0ca4153b9530e8b1a33bf64)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:54.770883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f26b64-2a4e-40dc-b50e-a3076c971be0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:54.870489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 617/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 18:59:54 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/c32b1292d0c6436ba4669dbfd1c73ea4/roles/be6b18fac0ca4153b9530e8b1a33bf64 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 18:59:57.014605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db66cf-4831-4217-83df-2584435999d9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:57.015140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db66cf-4831-4217-83df-2584435999d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:57.015140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db66cf-4831-4217-83df-2584435999d9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:57.016559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 616/1233] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 18:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 18:59:57.021937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:57.022373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:57.022490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:57.041890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 18:59:57.043167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 18:59:57.109003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 618/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 18:59:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 18:59:57.377349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db66cf-4831-4217-83df-2584435999d9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:57.405056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:57.481811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db66cf-4831-4217-83df-2584435999d9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:57.514674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:57.515838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:57.516117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:57.516389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:57.526692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 18:59:57.532089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:57.533196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:59:57.534363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 617/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Sep 7 18:59:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:57.539220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:57.584192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:57.634273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9afec4-bb10-44e4-95c5-c7f5f6892928 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:57.667905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:57.668502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:57.668775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:57.669042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:57.669715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:57.670016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:57.671438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:57.679121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 619/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Sep 7 18:59:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:57.684999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:57.770024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:57.785500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d31f-d7f9-4e63-a513-ac77d1d4b3e9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:57.819280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:57.819898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:57.820175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:57.820449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:57.830894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:57.832309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:57.846391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:57.847674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:57.848579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:57.848579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:57.848579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:59:57.848579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 618/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:57.853305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:57.955157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7997f-0f4a-4a3f-9e4c-6e500479c02c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:57.964086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:57.989428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:57.990072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:57.990355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:57.990662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:57.991304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:57.991611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:57.992979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:58.005807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 620/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:58.011613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:58.111392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60440cb7-52fe-482d-9f94-e51507c4ac8b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:58.148615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:58.149240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:58.149564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:58.149838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:58.153062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:58.158492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:58.159948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 18:59:58.170952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:58.172642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 18:59:58.173581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 619/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:58.178662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:58.279614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39844f62-6e92-4fe0-acfa-0c24d007e0c0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:58.313743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:58.314338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:58.314641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:58.314915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:58.315683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:58.315980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:58.317402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:58.333884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 621/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Sep 7 18:59:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:58.339770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:58.441196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9169c-752a-4ee4-bf1f-9426f87b3ae3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:58.475118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:58.475707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:58.475986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:58.476261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:58.538180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:58.544473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:58.545916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:58.639888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 620/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 18:59:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 18:59:58.646275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:58.674481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:58.744849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698473c-fe2b-4716-9e34-87182c5b87da None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:58.781073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:58.781695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:58.781971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:58.782241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:58.807802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:58.807802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 18:59:58.808277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:58.809369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 18:59:58.810477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 18:59:58.810477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 18:59:58.810477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:58.810477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 622/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 18:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:58.815819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:58.909579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:58.916021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8af87-53e1-497f-906f-0a0bb99fc49c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:58.950216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:58.950836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:58.951115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:58.951386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:58.952039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:58.952326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:58.953710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:58.964802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 621/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 18:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:58.970765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:59.050843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 18:59:59.071709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990153-e866-4559-afb6-0c0e609533bd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:59.104455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:59.105071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:59.105386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:59.105661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:59.112834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:59.114274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 18:59:59.125384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 18:59:59.126295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 18:59:59.127304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 623/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 18:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 18:59:59.132577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:59.235031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c3e33-a8f2-4359-94b7-0b6a99dd56fb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:59.268057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:59.268727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:59.269000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:59.269268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:59.269949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:59.270249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:59.271722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69598da7-49bf-4f8d-b4b9-32a6ba11ad0f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:59.284184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 622/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 18:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 18:59:59.290295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 18:59:59.390886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ce397-e685-446d-bf7b-8b01a5c357ff None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 18:59:59.424484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 18:59:59.425080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 18:59:59.425391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 18:59:59.425664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 18:59:59.426374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] RBAC: Adding query filter params (role.id=be6b18fac0ca4153b9530e8b1a33bf64, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=c32b1292d0c6436ba4669dbfd1c73ea4) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 18:59:59.426710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 18:59:59.428131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f56b94-6b3c-46f5-bd97-a1da1d6934c3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 18:59:59.433804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 624/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 18:59:59 2022] GET /identity/v3/role_assignments?user.id=c32b1292d0c6436ba4669dbfd1c73ea4&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=be6b18fac0ca4153b9530e8b1a33bf64 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:01.655628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996a100-763a-4dcd-a938-591887c74a12 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:01.656169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996a100-763a-4dcd-a938-591887c74a12 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:01.656169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996a100-763a-4dcd-a938-591887c74a12 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:01.657498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 623/1247] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:01.662733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:01.663151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:01.663284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:01.670732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:01.685213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:01.686468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:01.751577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 625/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:02.019909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996a100-763a-4dcd-a938-591887c74a12 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:02.039990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:02.123271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996a100-763a-4dcd-a938-591887c74a12 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:02.157441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:02.158065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:02.158342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:02.158640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:02.168730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 19:00:02.174663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:02.175793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 19:00:02.176854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:02.181545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 624/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:02.182336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:02.281572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580d21a-a331-4fb2-8e79-deed7d2eaba7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:02.311294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:02.316197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:02.318775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:02.318775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:02.318775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:02.318775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:02.318775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:02.319663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:02.328333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 626/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:00:02.334307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:02.411124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:02.433854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f00dd6-fe93-495c-87e1-179642311a09 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:02.469623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:02.470249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:02.470529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:02.470833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:02.480534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:02.481964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 19:00:02.496499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:02.497527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 19:00:02.498475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 19:00:02.498475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:02.498475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:02.498722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 625/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:02.503686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:02.602540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b810973-6107-43d0-8606-b3f191e931a6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:02.635012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:02.635612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:02.635891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:02.636163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:02.636799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:02.637104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:02.638470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:02.650716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 627/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:02.657026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:02.756354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c5a5-c482-4fde-b465-503801243c80 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:02.782948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:02.790865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:02.791488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:02.791763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:02.792031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:02.799462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:02.800890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 19:00:02.812220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:02.813340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:02.814746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 626/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:02.819173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:02.874858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:02.920958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d50725-ce75-4e00-ab60-5dc289f08357 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:02.956151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:02.956751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:02.957028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:02.957301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:02.958019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:02.958333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:02.959781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:02.976727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 628/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:02.982886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:03.086238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88b0c1-e7b1-4ffc-b02f-5cca88723ab6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:03.118689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:03.119290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:03.119569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:03.119846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:03.176668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:03.189246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:03.190713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:03.292579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 627/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 19:00:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 19:00:03.299033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:03.400612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e5160-8761-43e1-8afa-39d3c5d48b66 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:03.413359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:03.433749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:03.434310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:03.434622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:03.434896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:03.445039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:03.448887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 19:00:03.461173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:03.462301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 19:00:03.463263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 19:00:03.463263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:03.463263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:03.463263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 629/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:03.468138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:03.556709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:03.568516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313a24e-317a-4fa1-ae5a-517e57816d20 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:03.601973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:03.602591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:03.602871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:03.603143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:03.603778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:03.604078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:03.605418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:03.616731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 628/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 19:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:03.622443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:03.722375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b326b72-d146-46c8-9bf2-aae5382b10cb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:03.755982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:03.756601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:03.756872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:03.757139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:03.763748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:03.765236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 19:00:03.776427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:03.777459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:03.778369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 630/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 19:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:03.784044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:03.885184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd270bef-82d0-4d79-a88c-3b4761618af6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:03.919508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:03.920114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:03.920394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:03.920663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:03.921927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:03.922227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:03.923671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62aedd-3c42-4c32-8dc8-41035c8b2143 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:03.935569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 629/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 19:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:03.941893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:03.968392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:04.042311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80aedf-116c-4e6a-8fcb-e5cfefd3bf48 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:04.080157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:04.080787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:04.081066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:04.081342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:04.082066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=c32b1292d0c6436ba4669dbfd1c73ea4) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:04.082372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:04.083856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b3048-9a08-4245-a0e3-1bb0ae47246f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:04.089292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 631/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 19:00:03 2022] GET /identity/v3/role_assignments?user.id=c32b1292d0c6436ba4669dbfd1c73ea4&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:06.266675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e1904-aef7-4791-839b-350016c1cb63 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:06.267149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e1904-aef7-4791-839b-350016c1cb63 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:06.267256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e1904-aef7-4791-839b-350016c1cb63 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:06.268560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 630/1261] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:06.273838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:06.274265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:06.274390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:06.293108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:06.294290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:06.359855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 632/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:06.630671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e1904-aef7-4791-839b-350016c1cb63 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:06.642112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:06.728750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e1904-aef7-4791-839b-350016c1cb63 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:06.761917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:06.762530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:06.762834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:06.763103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:06.773326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 19:00:06.778387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:06.779462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:06.780910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 631/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 19:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:06.786198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:06.804180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:06.886114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d64aa-b1d6-4a0b-b093-44d2530dc498 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:06.919704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:06.920299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:06.920602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:06.920874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:06.921528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:06.921826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:06.923211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:06.930930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 633/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 19:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:06.936725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.003026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:07.036177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56577250-83c8-4080-9ef8-c4a229ce526c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:07.070566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:07.071190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:07.071465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:07.071733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:07.082169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:07.083695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 19:00:07.098808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:07.099850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 19:00:07.101135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 19:00:07.101135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:07.101135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:07.101135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 632/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:07.105864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.187598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:07.206560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf770a6-f1eb-4874-b801-5479a5902974 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:07.239278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:07.239878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:07.240156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:07.240438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:07.241342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:07.241656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:07.243376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:07.256855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 634/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 19:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:07.262668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.366399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9e4af-fe13-47e2-ad7f-d23167692ed5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:07.385847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:07.400482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:07.401104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:07.401381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:07.401651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:07.409023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:07.410437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 19:00:07.421285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:07.422428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:07.423505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 633/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:07.428464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.528405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04282f9-e237-414e-8a78-494e8f6b1bb8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:07.564165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:07.564767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:07.565054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:07.565326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:07.566082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:07.566401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:07.567846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:07.576580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:07.585317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 635/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Sep 7 19:00:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:07.591362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.691950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449c5e4-fb1c-47f9-8fc9-b443bb8ff81c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:07.724292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:07.724909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:07.725189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:07.725465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:07.736689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:07.739479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 19:00:07.752562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:07.753697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 19:00:07.754629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 19:00:07.754629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:07.754629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:07.754629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 634/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:07.759654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.857820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a06f6-9311-43a2-a841-b80cb78663b8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:07.892006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:07.892597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:07.892875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:07.893145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:07.893769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:07.894070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:07.895450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:07.906768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 636/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 19:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:07.912829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:07.944862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:08.015398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f7ed-0c1e-4d64-a09e-6e848f7b8744 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:08.048254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:08.048873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:08.049151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:08.049423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:08.056022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:08.057500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 19:00:08.069131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:08.070169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:08.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 635/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 19:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:08.076326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:08.134746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:08.176363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e26d2-6646-4bfd-8241-a4fc4e467960 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:08.212174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:08.212819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:08.213145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:08.213420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:08.214103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:08.214410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:08.215877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e75d04-eb25-4c86-a59b-4594f860ee01 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:08.230267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 637/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 19:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:08.237311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:08.328347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:08.336788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755785df-6471-42c7-a607-bc52dada22d1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:08.368963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277b3b2-4350-4e48-a111-970d34cd1292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:08.369602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b3b2-4350-4e48-a111-970d34cd1292 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:08.369870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b3b2-4350-4e48-a111-970d34cd1292 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:08.370141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b3b2-4350-4e48-a111-970d34cd1292 None admin] PATH_INFO: `/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/c32b1292d0c6436ba4669dbfd1c73ea4/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:08.372126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b3b2-4350-4e48-a111-970d34cd1292 None admin] RBAC: Authorizing `identity:create_grant(project_id=0384adf02c8a481aaa2816ff8339e704, user_id=c32b1292d0c6436ba4669dbfd1c73ea4, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:08.374293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b3b2-4350-4e48-a111-970d34cd1292 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:08.452760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 636/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Sep 7 19:00:08 2022] PUT /identity/v3/projects/0384adf02c8a481aaa2816ff8339e704/users/c32b1292d0c6436ba4669dbfd1c73ea4/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:00:10.607497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e5074-23eb-42aa-861c-d858e7eeb438 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:10.608564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e5074-23eb-42aa-861c-d858e7eeb438 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:10.608564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e5074-23eb-42aa-861c-d858e7eeb438 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:10.609926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 638/1274] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:10.615377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:10.615770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:10.615922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:10.635112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:10.636298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:10.702324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 637/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:10.976339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e5074-23eb-42aa-861c-d858e7eeb438 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:11.078269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e5074-23eb-42aa-861c-d858e7eeb438 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:11.095286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:11.111807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:11.112420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:11.112693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:11.112959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:11.124065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 19:00:11.129205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:11.130421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:11.131331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 639/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Sep 7 19:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:11.136332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:11.217497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:11.235188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9241d48-d26a-40ad-8027-742162a7104b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:11.268509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:11.269109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:11.269388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:11.269659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:11.270317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:11.270651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:11.272001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:11.279959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 638/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Sep 7 19:00:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:11.285836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:11.386431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea1c2d-19cc-4348-8449-756d9a44fff2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:11.419671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:11.420811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:11.421086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:11.421351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:11.431155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:11.432626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 19:00:11.447411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:11.448500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 19:00:11.449772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 19:00:11.449772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:11.449772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:11.449772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 640/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:11.454356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:11.551963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0310607-daee-4b5d-b189-3bfeb2683a47 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:11.584741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:11.585335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:11.585610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:11.585878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:11.586506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:11.586832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:11.588198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:11.600067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 639/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 19:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:11.605735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:11.636149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:11.706047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870adc2-6dc9-46f6-aa8a-e6f28dc5536d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:11.739604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:11.740217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:11.740494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:11.740765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:11.747949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:11.749481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 19:00:11.760432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:11.761514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:11.762826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 641/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:11.767445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:11.771206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:11.868287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35142ec7-e002-4a8a-802a-dfa76a698370 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:11.901433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:11.902030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:11.902326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:11.902608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:11.903327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:11.903630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:11.905528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:11.922797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 640/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Sep 7 19:00:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:11.928632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:12.019693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:12.029153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407055a-126e-497f-841a-dac506e3d6ef None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:12.062223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:12.062851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:12.063129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:12.063401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:12.129226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:12.130632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:12.220022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:12.230137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 642/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Sep 7 19:00:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 19:00:12.237075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:12.298429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:12.336286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e423d1-19fb-4b5a-81e4-4aaebaebed0f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:12.369273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:12.369888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:12.370164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:12.370433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:12.380758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:12.382117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 19:00:12.397736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:12.400690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 19:00:12.401845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 19:00:12.401845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 19:00:12.401845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:12.401845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 641/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:12.404965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:12.502981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed043a-0d77-4498-a7cd-0cd48a3f6ea4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:12.537712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:12.538309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:12.538610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:12.538887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:12.539526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:12.539830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:12.541196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:12.552272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 643/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Sep 7 19:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:12.557911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:12.658052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7a5a5-4222-4eb4-839d-65998c9a33f5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:12.671322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:12.691121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:12.691744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:12.692021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:12.692294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:12.698743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:12.700186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 19:00:12.711816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:12.712908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 07 19:00:12.714029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 642/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Sep 7 19:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:12.718913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:12.768351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:12.818618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071b85-b8af-4cfc-8016-79dd5147213e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:12.851718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:12.852325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:12.852590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:12.852859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:12.853534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:12.853835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:12.855375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d94c3-919a-4e11-8f8e-949b561b0540 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:12.867586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 644/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Sep 7 19:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:12.874691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:12.972819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750452b0-664b-4577-bbc4-4ef0eec200ed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:13.006568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:13.007181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:13.007460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:13.007732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:13.008463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] RBAC: Adding query filter params (role.id=399ebd44d9c74dcc9954c0828ffd63c6, scope.project.id=0384adf02c8a481aaa2816ff8339e704, user.id=c32b1292d0c6436ba4669dbfd1c73ea4) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:13.008752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:13.010236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68893e9f-db7a-4a61-887b-4b066a14c077 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:13.015887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 643/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Sep 7 19:00:12 2022] GET /identity/v3/role_assignments?user.id=c32b1292d0c6436ba4669dbfd1c73ea4&scope.project.id=0384adf02c8a481aaa2816ff8339e704&role.id=399ebd44d9c74dcc9954c0828ffd63c6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:00:15.176722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5fd97-c28c-486d-a34d-bfc716fdd992 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:15.177259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5fd97-c28c-486d-a34d-bfc716fdd992 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:15.177387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5fd97-c28c-486d-a34d-bfc716fdd992 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:15.178750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 645/1288] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:15.183804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:15.184281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:15.184506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:15.203008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:15.204190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:15.234328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:15.271068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 644/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:15.539418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5fd97-c28c-486d-a34d-bfc716fdd992 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:15.638464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5fd97-c28c-486d-a34d-bfc716fdd992 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:15.649893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:15.673446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:15.674055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:15.674398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:15.674683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:15.675254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:15.676599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:15.685668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:15.692509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:15.693412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 646/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 19:00:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:15.694249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:15.791178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319bdeb-eb55-4d83-95dc-aa23291e7985 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:15.823532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:15.824132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:15.824415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:15.824688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:15.825310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:15.825609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:15.827493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfc99f-450e-454d-b213-206ef0a943c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:15.832098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 645/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Sep 7 19:00:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:15.838032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:15.936645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71aba3-8c80-4350-b9fa-c565dadb784a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:15.969977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:15.970615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:15.970891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:15.971159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:15.971772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:15.973374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:15.973522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56586677-8f03-4264-8fdd-eb48750f0689 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:15.977710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 647/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Sep 7 19:00:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:18.132817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef06aac-49e6-4937-9351-cbd422c9941b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:18.133408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef06aac-49e6-4937-9351-cbd422c9941b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:18.133408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef06aac-49e6-4937-9351-cbd422c9941b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:18.134714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 646/1293] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:18.139873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe33e49-e71d-4d97-8269-7a60825add5b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:18.140251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe33e49-e71d-4d97-8269-7a60825add5b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:18.140474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe33e49-e71d-4d97-8269-7a60825add5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:18.158715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7fe33e49-e71d-4d97-8269-7a60825add5b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:18.159916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7fe33e49-e71d-4d97-8269-7a60825add5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:18.189493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:18.226085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 648/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:18.505450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef06aac-49e6-4937-9351-cbd422c9941b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:18.536694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:18.603895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef06aac-49e6-4937-9351-cbd422c9941b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:18.637251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6e58b-a0a2-4f16-850f-78b47a00e964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:18.637847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6e58b-a0a2-4f16-850f-78b47a00e964 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:18.638123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6e58b-a0a2-4f16-850f-78b47a00e964 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:18.638392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6e58b-a0a2-4f16-850f-78b47a00e964 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:18.639133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6e58b-a0a2-4f16-850f-78b47a00e964 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:18.640492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6e58b-a0a2-4f16-850f-78b47a00e964 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:18.726403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 647/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Sep 7 19:00:18 2022] POST /identity/v3/services => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:00:20.890566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7394b-175f-4a3e-83f2-d4e39d8de97a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:20.891126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7394b-175f-4a3e-83f2-d4e39d8de97a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:20.891126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7394b-175f-4a3e-83f2-d4e39d8de97a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:20.892454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 649/1296] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:20.897597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:20.898013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:20.898158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:20.917637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:20.918860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:20.989922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 648/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:21.255478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7394b-175f-4a3e-83f2-d4e39d8de97a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:21.356498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7394b-175f-4a3e-83f2-d4e39d8de97a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:21.392264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:21.393329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:21.393872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:21.394352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:21.395533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:21.396945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:21.406737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:21.408023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:21.408965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 650/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 19:00:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:21.413963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:21.432010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:21.508891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43821e-1bc6-48d7-abe3-998145568c9c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:21.543671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:21.544495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:21.544770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:21.545085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:21.545707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:21.546006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:21.547382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:21.552301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 649/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Sep 7 19:00:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:21.558335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:21.592336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:21.660147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccf82a-9ae0-44d1-95ba-364086a745c8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:21.693767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:21.694370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:21.694666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:21.694943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:21.695636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] RBAC: Adding query filter params (interface=public, service_id=45d8aa33e7004c3cbfa1f05d357834ae, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:21.695942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:21.697459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fddfc8-5a95-40aa-8811-710cd1d5fda3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:21.703028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 651/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Sep 7 19:00:21 2022] GET /identity/v3/endpoints?service_id=45d8aa33e7004c3cbfa1f05d357834ae&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:21.708783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:21.788260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:21.807189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed0c0c-96fb-4055-9c38-3522cf907f2e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:21.841397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4cc5e-16ba-4c97-998d-91c01ee2df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:21.846312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4cc5e-16ba-4c97-998d-91c01ee2df70 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:21.846312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4cc5e-16ba-4c97-998d-91c01ee2df70 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:21.846312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4cc5e-16ba-4c97-998d-91c01ee2df70 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:21.846312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4cc5e-16ba-4c97-998d-91c01ee2df70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:21.846312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4cc5e-16ba-4c97-998d-91c01ee2df70 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:21.851219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 650/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Sep 7 19:00:21 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:00:24.025722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d88eb-dba2-48ea-a275-22a0dbfab58d None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:24.026283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d88eb-dba2-48ea-a275-22a0dbfab58d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:24.026283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d88eb-dba2-48ea-a275-22a0dbfab58d None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:24.027633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 652/1302] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:00:24.032912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:24.033356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:24.033460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:24.051714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:00:24.052889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:00:24.117808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 651/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:00:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 19:00:24.382835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d88eb-dba2-48ea-a275-22a0dbfab58d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:24.476447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:24.482877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d88eb-dba2-48ea-a275-22a0dbfab58d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:24.515841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:24.516456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:24.516730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:24.516996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:24.517570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:24.518950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 19:00:24.527675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 19:00:24.528874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Sep 07 19:00:24.530067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 653/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Sep 7 19:00:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 19:00:24.534778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:24.608380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:24.630315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b5f03-b154-4f32-9f7d-1bc7e605a530 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:24.663501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:24.664038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:24.664319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:24.664592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:24.665215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:00:24.665517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:24.666904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64536ab-801e-4696-a123-0d1d55b17b83 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:24.672806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 652/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Sep 7 19:00:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:00:24.678627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:00:24.780503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893acf5b-cf69-409d-b4e6-9bb756cafc8c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:00:24.813808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76a6c6-b0de-4a3d-bbf0-3770dfde4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:00:24.814399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a6c6-b0de-4a3d-bbf0-3770dfde4e6f None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:00:24.814703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a6c6-b0de-4a3d-bbf0-3770dfde4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:00:24.814977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a6c6-b0de-4a3d-bbf0-3770dfde4e6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:00:24.815643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76a6c6-b0de-4a3d-bbf0-3770dfde4e6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:00:24.817006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76a6c6-b0de-4a3d-bbf0-3770dfde4e6f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:00:24.909030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 654/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:00:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:00:34.669926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=30 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:00:34.904906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:11.280440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b98c-4f3b-4d18-8087-cc9dc93d6428 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:11.280902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b98c-4f3b-4d18-8087-cc9dc93d6428 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:11.280982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b98c-4f3b-4d18-8087-cc9dc93d6428 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:11.287998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c6e27-1f76-42f8-a2e5-163ea0e37b62 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:11.288426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c6e27-1f76-42f8-a2e5-163ea0e37b62 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:11.288602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c6e27-1f76-42f8-a2e5-163ea0e37b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:11.297829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 653/1307] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:11.379135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f26c6e27-1f76-42f8-a2e5-163ea0e37b62 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:11.381224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f26c6e27-1f76-42f8-a2e5-163ea0e37b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:11.527677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 655/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:06:11.824659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d257-9e37-4e78-8f0f-38735e442581 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:11.825013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d257-9e37-4e78-8f0f-38735e442581 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:11.825248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d257-9e37-4e78-8f0f-38735e442581 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:11.827131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 654/1309] 10.222.0.46 () {54 vars in 965 bytes} [Wed Sep 7 19:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:11.832622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791730a-f8ab-4063-9436-b131e6690cb7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:11.833555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791730a-f8ab-4063-9436-b131e6690cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:11.833895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791730a-f8ab-4063-9436-b131e6690cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:11.928476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b791730a-f8ab-4063-9436-b131e6690cb7 None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:11.929689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b791730a-f8ab-4063-9436-b131e6690cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:11.953715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:12.016537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 656/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Wed Sep 7 19:06:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:12.030527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2d257-9e37-4e78-8f0f-38735e442581 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:12.032293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5bb2d257-9e37-4e78-8f0f-38735e442581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:12.158888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2d257-9e37-4e78-8f0f-38735e442581 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:12.253805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:12.254820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:12.255771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:12.256251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:12.258247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:12.320267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:12.378127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:12.379590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54bc06-52f9-45ca-9a1c-4d2783d6b10e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:12.399762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 655/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:14.871070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283412f-96aa-46fd-9987-002dfb90355a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:14.871584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283412f-96aa-46fd-9987-002dfb90355a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:14.871584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283412f-96aa-46fd-9987-002dfb90355a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:14.873129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 657/1312] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:14.878400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e214c-c78a-47d4-9750-e7b1f4f9b695 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:14.878835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e214c-c78a-47d4-9750-e7b1f4f9b695 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:14.879016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e214c-c78a-47d4-9750-e7b1f4f9b695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:14.961832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-244e214c-c78a-47d4-9750-e7b1f4f9b695 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:14.963066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-244e214c-c78a-47d4-9750-e7b1f4f9b695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:15.065225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 656/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:15.349767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283412f-96aa-46fd-9987-002dfb90355a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:15.351401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c283412f-96aa-46fd-9987-002dfb90355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:15.466595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283412f-96aa-46fd-9987-002dfb90355a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:15.553569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:15.556089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:15.556756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:15.557026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:15.557286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:15.559003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:15.669497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:15.671068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8900a6-57e5-43a1-99b1-898a9e2759a8 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:15.689197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 658/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:18.346893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9eff1-f3c1-47bd-bcd4-a5e2c152e164 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:18.347419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9eff1-f3c1-47bd-bcd4-a5e2c152e164 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:18.347419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9eff1-f3c1-47bd-bcd4-a5e2c152e164 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:18.349422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 657/1315] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:18.354690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee464593-8787-4de9-8be1-e5ec31ca5a2e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:18.355130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee464593-8787-4de9-8be1-e5ec31ca5a2e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:18.355239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee464593-8787-4de9-8be1-e5ec31ca5a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:18.421378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ee464593-8787-4de9-8be1-e5ec31ca5a2e None None] MFA Rules not processed for user `87769f448fc54978888c2c57ac8ab26a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:18.422586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ee464593-8787-4de9-8be1-e5ec31ca5a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:18.472815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:18.516321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 659/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:18.802087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9eff1-f3c1-47bd-bcd4-a5e2c152e164 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:18.803691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a2e9eff1-f3c1-47bd-bcd4-a5e2c152e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:18.813316 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:18.907036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9eff1-f3c1-47bd-bcd4-a5e2c152e164 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:18.978405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:18.979026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:18.979301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:18.979564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:18.981132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:19.041338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:19.087228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:19.088653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c271-0cfb-47ef-9d15-4190e6c0727d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:19.105937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 658/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:22.032281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e9a3f-193c-419b-b918-60b4ebc404b6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:22.032892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e9a3f-193c-419b-b918-60b4ebc404b6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:22.032892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e9a3f-193c-419b-b918-60b4ebc404b6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:22.034171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 660/1318] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 19:06:22.040374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a6015-66ff-4793-b893-1120d0ac8cdd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:22.040828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a6015-66ff-4793-b893-1120d0ac8cdd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:22.040946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a6015-66ff-4793-b893-1120d0ac8cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:22.105773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-985a6015-66ff-4793-b893-1120d0ac8cdd None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:22.107004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-985a6015-66ff-4793-b893-1120d0ac8cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:22.202050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 659/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:22.493065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e9a3f-193c-419b-b918-60b4ebc404b6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:22.494630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3c2e9a3f-193c-419b-b918-60b4ebc404b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:22.596741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e9a3f-193c-419b-b918-60b4ebc404b6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:22.670122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:22.670741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:22.671017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:22.671278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:22.672837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:22.680107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:22.779656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:22.781000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81813684-6ee0-41e6-8e6d-819ad4edd226 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:22.798992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 661/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:26.029158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430ff5-efdc-4f34-884d-49bfcd552994 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:26.029735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430ff5-efdc-4f34-884d-49bfcd552994 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:26.029735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430ff5-efdc-4f34-884d-49bfcd552994 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:26.031128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 660/1321] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:26.037865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2d89-2748-4c70-85a6-91e6bc184c4d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:26.038417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2d89-2748-4c70-85a6-91e6bc184c4d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:26.038514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2d89-2748-4c70-85a6-91e6bc184c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:26.110168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-357e2d89-2748-4c70-85a6-91e6bc184c4d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:26.111459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-357e2d89-2748-4c70-85a6-91e6bc184c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:26.183426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:26.226055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 662/1322] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:26.508379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d430ff5-efdc-4f34-884d-49bfcd552994 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:26.510016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2d430ff5-efdc-4f34-884d-49bfcd552994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:26.530597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:26.617382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d430ff5-efdc-4f34-884d-49bfcd552994 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:26.693444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:26.694164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:26.694447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:26.694940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:26.696536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:26.770135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:26.808555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:26.810008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8657dae-acfc-4310-8478-cd9abb181463 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:26.827749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 661/1323] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:29.119025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056bcc-7a2e-44c0-b06d-93a4231ac3de None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:29.119643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056bcc-7a2e-44c0-b06d-93a4231ac3de None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:29.119643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056bcc-7a2e-44c0-b06d-93a4231ac3de None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:29.121056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 663/1324] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:29.126349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101ccb0-c646-45cc-bb43-dca38618de4b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:29.126842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101ccb0-c646-45cc-bb43-dca38618de4b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:29.126952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101ccb0-c646-45cc-bb43-dca38618de4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:29.202232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f101ccb0-c646-45cc-bb43-dca38618de4b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:29.203463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f101ccb0-c646-45cc-bb43-dca38618de4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:29.303536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 662/1325] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:29.586932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0056bcc-7a2e-44c0-b06d-93a4231ac3de None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:29.588452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d0056bcc-7a2e-44c0-b06d-93a4231ac3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:29.693398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0056bcc-7a2e-44c0-b06d-93a4231ac3de None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:29.768892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:29.769496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:29.770290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:29.770571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:29.772176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:29.789643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:29.881916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:29.883302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc99ad-a5b7-4f5c-8f24-63a64b48631b service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:29.902405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 664/1326] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:32.133782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbaa40-dbb4-4017-adbc-667e83eecd3c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:32.134309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbaa40-dbb4-4017-adbc-667e83eecd3c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:32.134309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbaa40-dbb4-4017-adbc-667e83eecd3c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:32.135777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 663/1327] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:32.141002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4be1bb-6259-4720-96f2-3ce85ba034fd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:32.141423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4be1bb-6259-4720-96f2-3ce85ba034fd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:32.141534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4be1bb-6259-4720-96f2-3ce85ba034fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:32.208016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ae4be1bb-6259-4720-96f2-3ce85ba034fd None None] MFA Rules not processed for user `87769f448fc54978888c2c57ac8ab26a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:32.209216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ae4be1bb-6259-4720-96f2-3ce85ba034fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:32.275862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:32.308041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 665/1328] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:32.593395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbaa40-dbb4-4017-adbc-667e83eecd3c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:32.595049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61fbaa40-dbb4-4017-adbc-667e83eecd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:32.626346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:32.708634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbaa40-dbb4-4017-adbc-667e83eecd3c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:32.790956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:32.791572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:32.791854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:32.792125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:32.793740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:32.876397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:32.909123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:32.910596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9c55f-668d-47cd-9e42-66d51ee61de3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:32.930490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 664/1329] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:35.515264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75914965-31be-4d14-ab16-a0ff83090554 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:35.515792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75914965-31be-4d14-ab16-a0ff83090554 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:35.515792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75914965-31be-4d14-ab16-a0ff83090554 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:35.517087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 666/1330] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:35.523557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f468c-1d6e-4470-87d0-cf9545f670d9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:35.524094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f468c-1d6e-4470-87d0-cf9545f670d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:35.524094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f468c-1d6e-4470-87d0-cf9545f670d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:35.591116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2a3f468c-1d6e-4470-87d0-cf9545f670d9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:35.598370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2a3f468c-1d6e-4470-87d0-cf9545f670d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:35.688721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 665/1331] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:06:35.970705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75914965-31be-4d14-ab16-a0ff83090554 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:35.972271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-75914965-31be-4d14-ab16-a0ff83090554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:36.074582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75914965-31be-4d14-ab16-a0ff83090554 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:36.148540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:36.149140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:36.149416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:36.149685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:36.151312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:36.178523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:36.263035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:36.264472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fc771-ba8f-4f9b-a71b-c3291354f6b2 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:36.282682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 667/1332] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:38.475256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccdd3a-e7ce-43c5-8f45-000a83d3dba5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:38.475795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccdd3a-e7ce-43c5-8f45-000a83d3dba5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:38.475795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccdd3a-e7ce-43c5-8f45-000a83d3dba5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:38.477145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 666/1333] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:38.482331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732843-c89d-44ff-98da-8ad03a4cb745 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:38.482797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732843-c89d-44ff-98da-8ad03a4cb745 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:38.482900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732843-c89d-44ff-98da-8ad03a4cb745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:38.548136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5d732843-c89d-44ff-98da-8ad03a4cb745 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:38.549346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5d732843-c89d-44ff-98da-8ad03a4cb745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:38.624345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:38.646746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 668/1334] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:06:38.942639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccdd3a-e7ce-43c5-8f45-000a83d3dba5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:38.944176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b2ccdd3a-e7ce-43c5-8f45-000a83d3dba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:38.984128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:39.047946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccdd3a-e7ce-43c5-8f45-000a83d3dba5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:39.136724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:39.137332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:39.137603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:39.137877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:39.139515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:39.221913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:39.247525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:39.248960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebd48e-bc14-4490-b103-67c5a05f67a8 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:39.267417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 667/1335] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:41.935526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfecff7-8b0a-44d0-8d3c-34edac111952 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:41.936076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfecff7-8b0a-44d0-8d3c-34edac111952 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:41.936076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfecff7-8b0a-44d0-8d3c-34edac111952 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:41.937306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 669/1336] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:41.942617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e38ed-2d1b-457c-8616-3f61f564d884 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:41.943069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e38ed-2d1b-457c-8616-3f61f564d884 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:41.943194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e38ed-2d1b-457c-8616-3f61f564d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:42.007681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f27e38ed-2d1b-457c-8616-3f61f564d884 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:42.008880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f27e38ed-2d1b-457c-8616-3f61f564d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:42.104092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 668/1337] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:42.386187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfecff7-8b0a-44d0-8d3c-34edac111952 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:42.387769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2cfecff7-8b0a-44d0-8d3c-34edac111952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:42.498838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfecff7-8b0a-44d0-8d3c-34edac111952 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:42.571594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:42.572187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:42.572467 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:42.572739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:42.574338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:42.609137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:42.682621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:42.684184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e04b2-342c-4589-98a3-16fbff8f8ec5 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:42.703091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 670/1338] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:46.974906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a20a-1fce-473a-b2c4-a9052fbee748 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:46.975463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a20a-1fce-473a-b2c4-a9052fbee748 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:46.975463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a20a-1fce-473a-b2c4-a9052fbee748 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:46.976784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 669/1339] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:46.982084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c687510-13d1-489d-9ad2-df0fff4a1e09 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:46.982487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c687510-13d1-489d-9ad2-df0fff4a1e09 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:46.982706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c687510-13d1-489d-9ad2-df0fff4a1e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:47.049584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9c687510-13d1-489d-9ad2-df0fff4a1e09 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:47.051453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9c687510-13d1-489d-9ad2-df0fff4a1e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:47.136971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:47.147890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 671/1340] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:47.448252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d313e-8045-4d22-bf70-acbcb32fc0be None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:47.448531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d313e-8045-4d22-bf70-acbcb32fc0be None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:47.448844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d313e-8045-4d22-bf70-acbcb32fc0be None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:47.450190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 670/1341] 10.222.0.46 () {54 vars in 965 bytes} [Wed Sep 7 19:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:47.456240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a496f-fc01-4ad3-b71d-5c8184ad7ff7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:47.456528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a496f-fc01-4ad3-b71d-5c8184ad7ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:47.457292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a496f-fc01-4ad3-b71d-5c8184ad7ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:47.533538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-624a496f-fc01-4ad3-b71d-5c8184ad7ff7 None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:47.534769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-624a496f-fc01-4ad3-b71d-5c8184ad7ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:47.612290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 672/1342] 10.222.0.46 () {58 vars in 1062 bytes} [Wed Sep 7 19:06:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:47.621488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d313e-8045-4d22-bf70-acbcb32fc0be None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:47.623200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca1d313e-8045-4d22-bf70-acbcb32fc0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:47.672940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:47.728456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d313e-8045-4d22-bf70-acbcb32fc0be None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:47.803675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:47.804285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:47.804563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:47.804834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:47.806448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:47.900908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:47.919575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:47.921046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223714f-89c4-4342-bf69-e0fd6bde3b10 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:47.942798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 671/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:51.551101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2649d-3063-41c8-a573-a7cbce0fff13 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:51.551623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2649d-3063-41c8-a573-a7cbce0fff13 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:51.551808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2649d-3063-41c8-a573-a7cbce0fff13 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:51.553177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 673/1344] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:51.558398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceae4ab-f043-47a7-ae3a-a5c100ed7b5d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:51.558900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceae4ab-f043-47a7-ae3a-a5c100ed7b5d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:51.559077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceae4ab-f043-47a7-ae3a-a5c100ed7b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:51.638595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4ceae4ab-f043-47a7-ae3a-a5c100ed7b5d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:51.639890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4ceae4ab-f043-47a7-ae3a-a5c100ed7b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:51.742746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 672/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:52.034512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2649d-3063-41c8-a573-a7cbce0fff13 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:52.036115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cda2649d-3063-41c8-a573-a7cbce0fff13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:52.125927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:52.143205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2649d-3063-41c8-a573-a7cbce0fff13 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:52.217225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:52.218091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:52.218365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:52.218662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:52.220250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:52.331791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:52.333242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1edb7ff-5f92-4112-ab34-3404c3c665d1 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:52.352420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 674/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:06:56.871672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af027a09-bfeb-433a-b2ac-39470458dd38 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:56.872227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af027a09-bfeb-433a-b2ac-39470458dd38 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:56.872227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af027a09-bfeb-433a-b2ac-39470458dd38 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:56.874159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 673/1347] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:06:56.881051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84dd56-bada-4194-bcfa-12431a4bf4f5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:56.881051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84dd56-bada-4194-bcfa-12431a4bf4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:56.881051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84dd56-bada-4194-bcfa-12431a4bf4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:56.898035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:56.954261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9d84dd56-bada-4194-bcfa-12431a4bf4f5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:06:56.955534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d84dd56-bada-4194-bcfa-12431a4bf4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:57.065548 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 675/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:06:57.347676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af027a09-bfeb-433a-b2ac-39470458dd38 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:06:57.349205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-af027a09-bfeb-433a-b2ac-39470458dd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:57.406202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:57.451898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af027a09-bfeb-433a-b2ac-39470458dd38 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:06:57.525005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:06:57.525609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:06:57.525907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:06:57.526181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:06:57.527802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:06:57.633781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:06:57.637319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:06:57.638899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801235f-2b83-4e9a-a493-b187569bbe5f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:06:57.659521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 674/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:02.592256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd529b-96ca-4708-8fa9-5ec8fd9c15b1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:02.592759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd529b-96ca-4708-8fa9-5ec8fd9c15b1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:02.592853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd529b-96ca-4708-8fa9-5ec8fd9c15b1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:02.594622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 676/1350] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:02.601069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab9fc1-4f13-442b-85ff-86706f4c2015 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:02.601069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab9fc1-4f13-442b-85ff-86706f4c2015 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:02.601069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab9fc1-4f13-442b-85ff-86706f4c2015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:02.747928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-44ab9fc1-4f13-442b-85ff-86706f4c2015 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:02.749408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-44ab9fc1-4f13-442b-85ff-86706f4c2015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:02.903303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 675/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:07:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:07.064119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=59 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:07.087145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2b53a-3ae4-41fa-a4f0-9d1cf2ea2c58 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:07.087470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2b53a-3ae4-41fa-a4f0-9d1cf2ea2c58 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:07.087728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2b53a-3ae4-41fa-a4f0-9d1cf2ea2c58 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:07.089212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 677/1352] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:07.094159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71406e74-64a3-4086-933d-4858534bb1e0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:07.094644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71406e74-64a3-4086-933d-4858534bb1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:07.095358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71406e74-64a3-4086-933d-4858534bb1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:07.151971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:07.176865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-71406e74-64a3-4086-933d-4858534bb1e0 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:07.178198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-71406e74-64a3-4086-933d-4858534bb1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:07.305912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 676/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:07:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:08.520511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723d2d-6bfd-4941-9f26-6e91691c9df8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:08.521015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723d2d-6bfd-4941-9f26-6e91691c9df8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:08.521015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723d2d-6bfd-4941-9f26-6e91691c9df8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:08.522518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 678/1354] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:08.528131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946b24c-e208-4a14-b275-b14e38f8b172 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:08.528575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946b24c-e208-4a14-b275-b14e38f8b172 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:08.528814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946b24c-e208-4a14-b275-b14e38f8b172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:08.608117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5946b24c-e208-4a14-b275-b14e38f8b172 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:08.609334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5946b24c-e208-4a14-b275-b14e38f8b172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:08.652210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:08.718964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 677/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:07:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:12.192219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b555c9-b85f-46c0-ab89-254c71729120 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:12.192756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b555c9-b85f-46c0-ab89-254c71729120 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:12.192756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b555c9-b85f-46c0-ab89-254c71729120 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:12.194017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 679/1356] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:12.199614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b833a0d-38f1-4793-8342-084c99738538 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:12.200066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b833a0d-38f1-4793-8342-084c99738538 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:12.200219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b833a0d-38f1-4793-8342-084c99738538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:12.271148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4b833a0d-38f1-4793-8342-084c99738538 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:12.272261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b833a0d-38f1-4793-8342-084c99738538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:12.380806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 678/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:12.679246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6800d-8e9f-45cb-928f-10eab66e0318 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:12.679904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6800d-8e9f-45cb-928f-10eab66e0318 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:12.680392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6800d-8e9f-45cb-928f-10eab66e0318 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:12.682324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 680/1358] 10.222.0.46 () {54 vars in 962 bytes} [Wed Sep 7 19:07:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:12.691633 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdee1e-cb14-4691-929f-1b348be1fbf7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:12.692163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdee1e-cb14-4691-929f-1b348be1fbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:12.692746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdee1e-cb14-4691-929f-1b348be1fbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:12.711123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:12.776444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b5bdee1e-cb14-4691-929f-1b348be1fbf7 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:12.778529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b5bdee1e-cb14-4691-929f-1b348be1fbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:12.910289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 679/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Sep 7 19:07:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:07:12.918439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6800d-8e9f-45cb-928f-10eab66e0318 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:12.921328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6cd6800d-8e9f-45cb-928f-10eab66e0318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:13.110271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6800d-8e9f-45cb-928f-10eab66e0318 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:13.194974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:13.249384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:13.250132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:13.250513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:13.250907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:13.253248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:13.356215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:13.358234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5afba4-2895-4238-ac8b-2c6eb0f0d990 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:13.378930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 681/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:07:15.766032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60422-9f42-4215-80a9-3a52d1d86d00 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:15.766483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60422-9f42-4215-80a9-3a52d1d86d00 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:15.766733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60422-9f42-4215-80a9-3a52d1d86d00 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:15.773776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724b399-fbd8-4d25-8d7b-a3274ce09c50 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:15.774146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724b399-fbd8-4d25-8d7b-a3274ce09c50 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:15.774321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724b399-fbd8-4d25-8d7b-a3274ce09c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:15.778814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 680/1361] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:15.845017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8724b399-fbd8-4d25-8d7b-a3274ce09c50 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:15.846209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8724b399-fbd8-4d25-8d7b-a3274ce09c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:15.885619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:15.951891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 682/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:07:16.240497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c60422-9f42-4215-80a9-3a52d1d86d00 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:16.242024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70c60422-9f42-4215-80a9-3a52d1d86d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:16.312471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:16.410733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c60422-9f42-4215-80a9-3a52d1d86d00 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:16.555447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:16.556352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:16.557523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:16.557523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:16.558734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:16.587743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:16.677392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:16.679038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d8d77-13d9-41ec-a369-29428211af91 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:16.699032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 681/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:19.360501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb118c-dd77-4e19-86a8-c078d0541582 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:19.361038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb118c-dd77-4e19-86a8-c078d0541582 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:19.361038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb118c-dd77-4e19-86a8-c078d0541582 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:19.363005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 683/1364] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:19.369810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959277e-6b6c-456a-89c5-40ca7a1ba675 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:19.370248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959277e-6b6c-456a-89c5-40ca7a1ba675 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:19.370408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959277e-6b6c-456a-89c5-40ca7a1ba675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:19.440453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b959277e-6b6c-456a-89c5-40ca7a1ba675 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:19.441708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b959277e-6b6c-456a-89c5-40ca7a1ba675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:19.543774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:19.546112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 682/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:19.840165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9eca6-8b2d-4ebf-aa7d-4e17484baa44 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:19.840567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9eca6-8b2d-4ebf-aa7d-4e17484baa44 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:19.840844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9eca6-8b2d-4ebf-aa7d-4e17484baa44 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:19.842586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 684/1366] 10.222.0.46 () {54 vars in 962 bytes} [Wed Sep 7 19:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:19.848206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf9bc5-91be-4bd6-aa09-a845b732cd60 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:19.848559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf9bc5-91be-4bd6-aa09-a845b732cd60 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:19.848953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf9bc5-91be-4bd6-aa09-a845b732cd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:19.928993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d0cf9bc5-91be-4bd6-aa09-a845b732cd60 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:19.930189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d0cf9bc5-91be-4bd6-aa09-a845b732cd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:20.052365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 683/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Sep 7 19:07:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:20.060203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9eca6-8b2d-4ebf-aa7d-4e17484baa44 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:20.061817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-64a9eca6-8b2d-4ebf-aa7d-4e17484baa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:20.226996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9eca6-8b2d-4ebf-aa7d-4e17484baa44 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:20.250015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:20.348662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:20.349376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:20.349723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:20.350063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:20.351853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:20.452307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:20.454002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31418d40-bf0e-4b3d-a342-ddf42a00619d service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:20.474354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 685/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:20.477111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61938822-1e93-42fa-909e-eaa7e889dc60 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:20.477589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61938822-1e93-42fa-909e-eaa7e889dc60 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:20.477589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61938822-1e93-42fa-909e-eaa7e889dc60 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:20.479084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 684/1369] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:20.484123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241665c6-f876-4b5c-adde-28a24dd9fc26 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:20.484409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241665c6-f876-4b5c-adde-28a24dd9fc26 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:20.484682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241665c6-f876-4b5c-adde-28a24dd9fc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:20.516672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:20.555335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-241665c6-f876-4b5c-adde-28a24dd9fc26 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:20.556576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-241665c6-f876-4b5c-adde-28a24dd9fc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:20.666632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 686/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:21.005242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d3e4-62c2-4632-bc67-7c6ec1a2484d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:21.005242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d3e4-62c2-4632-bc67-7c6ec1a2484d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:21.005242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d3e4-62c2-4632-bc67-7c6ec1a2484d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:21.007894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 685/1371] 10.222.0.46 () {54 vars in 964 bytes} [Wed Sep 7 19:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:21.010861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b231e-e0b7-4550-bfaf-d7e4b05073f0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:21.011154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b231e-e0b7-4550-bfaf-d7e4b05073f0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:21.011434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b231e-e0b7-4550-bfaf-d7e4b05073f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:21.101865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-922b231e-e0b7-4550-bfaf-d7e4b05073f0 None None] MFA Rules not processed for user `885f09bf588a442e931aae0da51d0ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:21.103126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-922b231e-e0b7-4550-bfaf-d7e4b05073f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:21.152553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:21.184325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 687/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Wed Sep 7 19:07:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:21.192395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74d3e4-62c2-4632-bc67-7c6ec1a2484d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:21.193891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e74d3e4-62c2-4632-bc67-7c6ec1a2484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:21.257327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:21.312870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74d3e4-62c2-4632-bc67-7c6ec1a2484d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:21.398655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:21.399386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:21.399754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:21.400016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:21.401624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:21.509096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:21.523147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:21.524199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e933b3-c364-4865-bd2c-722c4736a39c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:21.544926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 686/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:22.895977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a20e-9f63-47ec-9f1f-961282913ae1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:22.896504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a20e-9f63-47ec-9f1f-961282913ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:22.896504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a20e-9f63-47ec-9f1f-961282913ae1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:22.897908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 688/1374] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:22.903544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc86dc-be54-4ca8-99c1-5f13e5dadaf1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:22.904033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc86dc-be54-4ca8-99c1-5f13e5dadaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:22.910097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc86dc-be54-4ca8-99c1-5f13e5dadaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:22.971790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9fcc86dc-be54-4ca8-99c1-5f13e5dadaf1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:22.972925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9fcc86dc-be54-4ca8-99c1-5f13e5dadaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:23.072021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 687/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:23.369041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a20e-9f63-47ec-9f1f-961282913ae1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:23.370569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f981a20e-9f63-47ec-9f1f-961282913ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:23.524204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a20e-9f63-47ec-9f1f-961282913ae1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:23.578358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:23.648298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:23.648964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:23.649313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:23.649657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:23.651731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:23.753023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:23.754670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ac10-c9a0-4e21-9681-971edeede936 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:23.773480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 689/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:26.202159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ccb25-f5d7-46f6-b226-72d8a6571f1d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:26.202713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ccb25-f5d7-46f6-b226-72d8a6571f1d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:26.202838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ccb25-f5d7-46f6-b226-72d8a6571f1d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:26.204080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 688/1377] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:26.209444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4a81-1271-465b-a616-55021cac8bb9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:26.210171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4a81-1271-465b-a616-55021cac8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:26.210632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4a81-1271-465b-a616-55021cac8bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:26.284166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7f5e4a81-1271-465b-a616-55021cac8bb9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:26.285354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7f5e4a81-1271-465b-a616-55021cac8bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:26.293312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:26.390698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 690/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:26.678630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ccb25-f5d7-46f6-b226-72d8a6571f1d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:26.680132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d1ccb25-f5d7-46f6-b226-72d8a6571f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:26.750363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:26.840831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ccb25-f5d7-46f6-b226-72d8a6571f1d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:26.976152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:26.976913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:26.977279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:26.977642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:26.979422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:27.000959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:27.087312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:27.089005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec8c1a-a056-49ba-b626-1039ef0a5d14 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:27.111364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 689/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:29.492742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e0e87-0a63-4869-9608-ad523610f44a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:29.493279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e0e87-0a63-4869-9608-ad523610f44a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:29.493432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e0e87-0a63-4869-9608-ad523610f44a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:29.495354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 691/1380] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:29.500840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b7c8-62bc-4c78-95d4-7fecd7bcb154 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:29.501414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b7c8-62bc-4c78-95d4-7fecd7bcb154 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:29.501414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b7c8-62bc-4c78-95d4-7fecd7bcb154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:29.569047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8f67b7c8-62bc-4c78-95d4-7fecd7bcb154 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:29.570285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8f67b7c8-62bc-4c78-95d4-7fecd7bcb154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:29.687626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:29.696340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 690/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:29.988054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e0e87-0a63-4869-9608-ad523610f44a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:29.990793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b72e0e87-0a63-4869-9608-ad523610f44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:30.195203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:30.206005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e0e87-0a63-4869-9608-ad523610f44a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:30.335790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:30.336438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:30.336726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:30.337009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:30.339410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:30.440178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:30.441850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988b1e7-23d4-4155-b996-3a9ce9c7c544 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:30.452330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:30.460782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 692/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:33.347428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8d6c-eef8-4c1b-97db-3a8bf10d7b4a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:33.347964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8d6c-eef8-4c1b-97db-3a8bf10d7b4a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:33.348088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8d6c-eef8-4c1b-97db-3a8bf10d7b4a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:33.349424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 691/1383] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:33.356672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f56b5-e161-4f8f-bd8b-91385c57e0c3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:33.357192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f56b5-e161-4f8f-bd8b-91385c57e0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:33.357192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f56b5-e161-4f8f-bd8b-91385c57e0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:33.427251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5a9f56b5-e161-4f8f-bd8b-91385c57e0c3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:33.428456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5a9f56b5-e161-4f8f-bd8b-91385c57e0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:33.535275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 693/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:33.973195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8d6c-eef8-4c1b-97db-3a8bf10d7b4a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:33.975618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4c1d8d6c-eef8-4c1b-97db-3a8bf10d7b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:34.193908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8d6c-eef8-4c1b-97db-3a8bf10d7b4a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:34.279812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:34.337345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:34.338211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:34.338767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:34.339124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:34.341693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:34.446500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:34.448143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b905dd-24ec-43ec-9409-4e5207ee899c service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:34.466734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 692/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:36.606251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88be77f-0f8a-49fb-bf32-6a433c57a102 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:36.606820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88be77f-0f8a-49fb-bf32-6a433c57a102 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:36.606820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88be77f-0f8a-49fb-bf32-6a433c57a102 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:36.610383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 694/1386] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:36.613533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b8e15-2891-4865-b9b6-12ac075c2e8e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:36.613952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b8e15-2891-4865-b9b6-12ac075c2e8e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:36.614076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b8e15-2891-4865-b9b6-12ac075c2e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:36.685805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a34b8e15-2891-4865-b9b6-12ac075c2e8e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:36.687039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a34b8e15-2891-4865-b9b6-12ac075c2e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:36.722002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:36.786816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 693/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:37.072687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88be77f-0f8a-49fb-bf32-6a433c57a102 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:37.074231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f88be77f-0f8a-49fb-bf32-6a433c57a102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:37.114415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4b6d-6ad3-4fcb-bf40-6b1493549123 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:37.114720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4b6d-6ad3-4fcb-bf40-6b1493549123 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:37.115032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4b6d-6ad3-4fcb-bf40-6b1493549123 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:37.116385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 694/1388] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 19:07:37.121364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4867b-c851-40bc-8359-a4f7e7b2460e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:37.121649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4867b-c851-40bc-8359-a4f7e7b2460e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:37.121958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4867b-c851-40bc-8359-a4f7e7b2460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:37.152277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:37.191237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-17d4867b-c851-40bc-8359-a4f7e7b2460e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:37.192473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-17d4867b-c851-40bc-8359-a4f7e7b2460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:37.223889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88be77f-0f8a-49fb-bf32-6a433c57a102 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:37.292840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 695/1389] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:07:37.338792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:37.339458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:37.339741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:37.340011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:37.341599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:37.380885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:37.432380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:37.433984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fffa9-d4ba-4861-8f78-5ae21d70419b service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:37.452083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 695/1390] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:37.593515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b6371-a74f-4376-baa4-257d0b33b030 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:37.593729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b6371-a74f-4376-baa4-257d0b33b030 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:37.594002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b6371-a74f-4376-baa4-257d0b33b030 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:37.595877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 696/1391] 10.222.0.46 () {54 vars in 961 bytes} [Wed Sep 7 19:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:37.601205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d5ec5-5c4f-487a-ae63-53abde180195 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:37.601492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d5ec5-5c4f-487a-ae63-53abde180195 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:37.601773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d5ec5-5c4f-487a-ae63-53abde180195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:37.683446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-970d5ec5-5c4f-487a-ae63-53abde180195 None None] MFA Rules not processed for user `2f6d9e85ceed45e186421315737f9e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:37.684682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-970d5ec5-5c4f-487a-ae63-53abde180195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:37.770689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 696/1392] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Sep 7 19:07:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:37.780203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b6371-a74f-4376-baa4-257d0b33b030 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:37.781781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea0b6371-a74f-4376-baa4-257d0b33b030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:37.797132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:37.892741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b6371-a74f-4376-baa4-257d0b33b030 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:37.969430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:37.970042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:37.970323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:37.971177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:37.972769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:38.042533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:38.085605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:38.087075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e274d-87d7-4ee3-8799-112c790aaff0 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:38.107291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 697/1393] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:39.005654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941c3c-34a2-45e5-8e49-da50d3585829 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:39.006156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941c3c-34a2-45e5-8e49-da50d3585829 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:39.006263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941c3c-34a2-45e5-8e49-da50d3585829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:39.044098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:39.074501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5c941c3c-34a2-45e5-8e49-da50d3585829 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:39.076645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5c941c3c-34a2-45e5-8e49-da50d3585829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:39.165389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 697/1394] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:39.211995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58f4a-cef0-4258-b6c7-4e909dc3e9c6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:39.212500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58f4a-cef0-4258-b6c7-4e909dc3e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:39.212500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58f4a-cef0-4258-b6c7-4e909dc3e9c6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:39.214010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 698/1395] 10.222.0.46 () {54 vars in 943 bytes} [Wed Sep 7 19:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:39.220116 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a45c4-9d29-44d2-a690-3d7a06f3b00c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:39.221003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a45c4-9d29-44d2-a690-3d7a06f3b00c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:39.221346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a45c4-9d29-44d2-a690-3d7a06f3b00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:39.317243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-902a45c4-9d29-44d2-a690-3d7a06f3b00c None None] MFA Rules not processed for user `42374efb278647738c5834bdb80db3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:39.318647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-902a45c4-9d29-44d2-a690-3d7a06f3b00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:39.375825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:39.426344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 698/1396] 10.222.0.46 () {58 vars in 1040 bytes} [Wed Sep 7 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:07:39.435761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58f4a-cef0-4258-b6c7-4e909dc3e9c6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:39.437306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5be58f4a-cef0-4258-b6c7-4e909dc3e9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:39.574875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58f4a-cef0-4258-b6c7-4e909dc3e9c6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:39.612627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:39.682365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:39.682932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:39.683228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:39.683488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:39.685236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:39.775093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:39.776679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435fcce-a299-49f8-8119-672c98b375b7 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:39.785071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 699/1397] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:07:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:07:39.808183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d9159-0f16-4079-9426-39a61880e5c6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:39.808491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d9159-0f16-4079-9426-39a61880e5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:39.808766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d9159-0f16-4079-9426-39a61880e5c6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:39.810123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 699/1398] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:39.816088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c243-72d6-4423-ba3c-99437fdc5143 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:39.816370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c243-72d6-4423-ba3c-99437fdc5143 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:39.816644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c243-72d6-4423-ba3c-99437fdc5143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:39.882538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ab28c243-72d6-4423-ba3c-99437fdc5143 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:39.883763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab28c243-72d6-4423-ba3c-99437fdc5143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:39.902021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:39.984630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 700/1399] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:07:40.278628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d9159-0f16-4079-9426-39a61880e5c6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:40.284540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d0d9159-0f16-4079-9426-39a61880e5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:40.452882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d9159-0f16-4079-9426-39a61880e5c6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:40.495178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:40.584556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:40.585156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:40.585427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:40.585690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:40.587311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:40.692468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:40.694158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1176db-fd02-4232-bbf2-7e1ad04b209e service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:40.714620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 700/1400] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:43.096191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855f45-1c85-4078-8b43-71536dfcd3f5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:43.096794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855f45-1c85-4078-8b43-71536dfcd3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:43.096794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855f45-1c85-4078-8b43-71536dfcd3f5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:43.098253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 701/1401] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:43.104326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011158c-7f56-4264-bf70-375b09ee0332 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:43.106975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011158c-7f56-4264-bf70-375b09ee0332 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:43.106975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011158c-7f56-4264-bf70-375b09ee0332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:43.154970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:43.183137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4011158c-7f56-4264-bf70-375b09ee0332 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:43.184668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4011158c-7f56-4264-bf70-375b09ee0332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:43.300999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 701/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:43.588721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855f45-1c85-4078-8b43-71536dfcd3f5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:43.590494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-18855f45-1c85-4078-8b43-71536dfcd3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:43.751194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855f45-1c85-4078-8b43-71536dfcd3f5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:43.754173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:43.876673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:43.877212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:43.877467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:43.877729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:43.879358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:43.981890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:43.983642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c6f5a-89f4-4aa7-b3bf-e143c4f4c7b7 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:44.004039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 702/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:46.364075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9211f-c9dd-42d9-887c-3b369fabd5fa None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:46.364642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9211f-c9dd-42d9-887c-3b369fabd5fa None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:46.364642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9211f-c9dd-42d9-887c-3b369fabd5fa None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:46.369991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 702/1404] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:46.371351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac43bfa-eced-411a-b4cf-d88b1dfec244 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:46.371763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac43bfa-eced-411a-b4cf-d88b1dfec244 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:46.371954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac43bfa-eced-411a-b4cf-d88b1dfec244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:46.382329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:46.456442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ac43bfa-eced-411a-b4cf-d88b1dfec244 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:46.457740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ac43bfa-eced-411a-b4cf-d88b1dfec244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:46.564541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 703/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:07:46.859504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9211f-c9dd-42d9-887c-3b369fabd5fa None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:46.861549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c7a9211f-c9dd-42d9-887c-3b369fabd5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:46.980297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:47.023588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9211f-c9dd-42d9-887c-3b369fabd5fa None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:47.150118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:47.150687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:47.150978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:47.151268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:47.152812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:47.241912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:47.255538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:47.257164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47206e-6864-4d9e-8150-1e33ae833ccd service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:47.276934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 703/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:49.689539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0efb8-5692-40cb-9db8-6bef57d888b6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:49.690185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0efb8-5692-40cb-9db8-6bef57d888b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:49.690185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0efb8-5692-40cb-9db8-6bef57d888b6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:49.694919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 704/1407] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:07:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:07:49.696958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56c568-9375-4234-b249-c88ebb9004f5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:49.697377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56c568-9375-4234-b249-c88ebb9004f5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:49.697377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56c568-9375-4234-b249-c88ebb9004f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:49.772960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3e56c568-9375-4234-b249-c88ebb9004f5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:07:49.774278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3e56c568-9375-4234-b249-c88ebb9004f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:49.885713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 704/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:07:50.179538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0efb8-5692-40cb-9db8-6bef57d888b6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:07:50.181056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fee0efb8-5692-40cb-9db8-6bef57d888b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:50.278930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:50.349664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0efb8-5692-40cb-9db8-6bef57d888b6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:07:50.473506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:07:50.474025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:07:50.474357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:07:50.474662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:07:50.476260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:07:50.531206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:07:50.693502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:07:50.695880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a2ab7-3b8c-4295-ad83-abc26e61a119 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:07:50.717891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 705/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:07:59.884020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=76 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:00.715165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=24 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:03.422301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27bcf4-afdd-4f68-af68-3163a2260d8d None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:03.422939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27bcf4-afdd-4f68-af68-3163a2260d8d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:03.422939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27bcf4-afdd-4f68-af68-3163a2260d8d None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:03.424276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 705/1410] 10.222.0.46 () {54 vars in 918 bytes} [Wed Sep 7 19:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:03.429480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314351-c4ba-4fdb-970e-4c31c5667f81 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:03.429890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314351-c4ba-4fdb-970e-4c31c5667f81 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:03.430028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314351-c4ba-4fdb-970e-4c31c5667f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:03.505461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-28314351-c4ba-4fdb-970e-4c31c5667f81 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:03.506617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28314351-c4ba-4fdb-970e-4c31c5667f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:03.611833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 706/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Sep 7 19:08:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:03.626829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb57a0d-fb25-4964-a936-0439a58a8b78 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:03.627140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb57a0d-fb25-4964-a936-0439a58a8b78 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:03.627415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb57a0d-fb25-4964-a936-0439a58a8b78 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:03.628843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 706/1412] 10.222.0.46 () {54 vars in 918 bytes} [Wed Sep 7 19:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:03.633461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89e34-4926-471d-8ab2-cd3258f6550b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:03.633750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89e34-4926-471d-8ab2-cd3258f6550b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:03.634027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89e34-4926-471d-8ab2-cd3258f6550b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:03.707257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:03.709882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-15d89e34-4926-471d-8ab2-cd3258f6550b None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:03.711119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-15d89e34-4926-471d-8ab2-cd3258f6550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:03.825491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 707/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Sep 7 19:08:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:06.390563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf829aa-d3f2-42aa-9c9f-9435abcecbf3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:06.391066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf829aa-d3f2-42aa-9c9f-9435abcecbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:06.391169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf829aa-d3f2-42aa-9c9f-9435abcecbf3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:06.392490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:06.397689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5e8bb-a585-4a0d-94b4-ce5ebf8419d5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:06.398164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5e8bb-a585-4a0d-94b4-ce5ebf8419d5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:06.398270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5e8bb-a585-4a0d-94b4-ce5ebf8419d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:06.463840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d8a5e8bb-a585-4a0d-94b4-ce5ebf8419d5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:06.465055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d8a5e8bb-a585-4a0d-94b4-ce5ebf8419d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:06.519146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:06.565261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:06.852107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf829aa-d3f2-42aa-9c9f-9435abcecbf3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:06.853616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-edf829aa-d3f2-42aa-9c9f-9435abcecbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:06.999253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf829aa-d3f2-42aa-9c9f-9435abcecbf3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:07.070822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:07.112093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:07.112615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:07.112886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:07.113152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:07.114753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:07.201461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:07.203065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f831b-65a3-4fee-be0b-3d51628a218d service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:07.221919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 708/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:08.350740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05803629-c6fc-4813-8a6c-9f5945d4037a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:08.351306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05803629-c6fc-4813-8a6c-9f5945d4037a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:08.351306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05803629-c6fc-4813-8a6c-9f5945d4037a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:08.352621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:08.358819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e62a7-9251-49ab-8e8c-3dd2535b8643 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:08.359270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e62a7-9251-49ab-8e8c-3dd2535b8643 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:08.359368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e62a7-9251-49ab-8e8c-3dd2535b8643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:08.435517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0d8e62a7-9251-49ab-8e8c-3dd2535b8643 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:08.436751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0d8e62a7-9251-49ab-8e8c-3dd2535b8643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:08.463423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:08.533600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:08.682850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05803629-c6fc-4813-8a6c-9f5945d4037a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:08.684544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05803629-c6fc-4813-8a6c-9f5945d4037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:08.829659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05803629-c6fc-4813-8a6c-9f5945d4037a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:08.868344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:08.941119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:08.941718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:08.941993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:08.942262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:08.943871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:09.035575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:09.037141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53646122-5fa9-49c7-aa69-6379f9121855 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:09.054653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 710/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:08:10.189772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd7d2e-7450-41bc-8396-493632cd19da None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:10.190317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd7d2e-7450-41bc-8396-493632cd19da None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:10.190317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd7d2e-7450-41bc-8396-493632cd19da None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:10.191740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:10.197866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb40c07-7ff5-4aeb-b5e0-2b299bda0e52 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:10.198322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb40c07-7ff5-4aeb-b5e0-2b299bda0e52 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:10.198421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb40c07-7ff5-4aeb-b5e0-2b299bda0e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:10.255992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:10.278313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6cb40c07-7ff5-4aeb-b5e0-2b299bda0e52 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:10.279577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6cb40c07-7ff5-4aeb-b5e0-2b299bda0e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:10.375790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:10.527148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd7d2e-7450-41bc-8396-493632cd19da None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:10.528637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b6cd7d2e-7450-41bc-8396-493632cd19da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:10.674570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd7d2e-7450-41bc-8396-493632cd19da None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:10.686757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:10.788043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:10.788574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:10.788850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:10.789122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:10.790745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:10.879165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:10.880772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeabe0-f33b-465e-9259-880676bbe342 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:10.898633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 711/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:11.999226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5688ad-26b7-42cb-8018-c6311084f1b9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:11.999759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5688ad-26b7-42cb-8018-c6311084f1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:11.999759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5688ad-26b7-42cb-8018-c6311084f1b9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:12.001071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:12.007041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcf60b-b230-4062-b423-9ca0477614b7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:12.007501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcf60b-b230-4062-b423-9ca0477614b7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:12.007609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcf60b-b230-4062-b423-9ca0477614b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:12.026954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:12.083369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7edcf60b-b230-4062-b423-9ca0477614b7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:12.084562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7edcf60b-b230-4062-b423-9ca0477614b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:12.183485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:12.331583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5688ad-26b7-42cb-8018-c6311084f1b9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:12.333079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e5688ad-26b7-42cb-8018-c6311084f1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:12.441179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:12.477641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5688ad-26b7-42cb-8018-c6311084f1b9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:12.589215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:12.589744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:12.590024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:12.590585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:12.592177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:12.668788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:12.681561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:12.683184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681ec2a-3ec5-4b90-bb8c-7dfb48e98a58 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:12.702004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 713/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:08:13.808898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee78e7d-6185-4c3a-87fb-e8ba64376741 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:13.809471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee78e7d-6185-4c3a-87fb-e8ba64376741 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:13.809471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee78e7d-6185-4c3a-87fb-e8ba64376741 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:13.810807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:13.817073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc952e-18c6-409a-8a48-603d52247880 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:13.817486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc952e-18c6-409a-8a48-603d52247880 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:13.817606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc952e-18c6-409a-8a48-603d52247880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:13.891935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a7fc952e-18c6-409a-8a48-603d52247880 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:13.893118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a7fc952e-18c6-409a-8a48-603d52247880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:13.987817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:14.137841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee78e7d-6185-4c3a-87fb-e8ba64376741 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:14.149043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2ee78e7d-6185-4c3a-87fb-e8ba64376741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:14.229108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:14.290255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee78e7d-6185-4c3a-87fb-e8ba64376741 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:14.403362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:14.403892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:14.404171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:14.404440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:14.406009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:14.453533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:14.493332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:14.494946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e09198-a41f-417f-8c4d-c972c3d90bce service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:14.513512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 714/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:15.626349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2fe76-7935-413b-8c0d-c78233361a36 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:15.626945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2fe76-7935-413b-8c0d-c78233361a36 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:15.626945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2fe76-7935-413b-8c0d-c78233361a36 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:15.628283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 19:08:15.634485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c173bb-eb4c-4aec-8f6a-7cc63153b9a5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:15.634979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c173bb-eb4c-4aec-8f6a-7cc63153b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:15.635090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c173bb-eb4c-4aec-8f6a-7cc63153b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:15.709947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a6c173bb-eb4c-4aec-8f6a-7cc63153b9a5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:15.711193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6c173bb-eb4c-4aec-8f6a-7cc63153b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:15.806446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:15.956038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2fe76-7935-413b-8c0d-c78233361a36 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:15.957445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0fc2fe76-7935-413b-8c0d-c78233361a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:16.005252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:16.100480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2fe76-7935-413b-8c0d-c78233361a36 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:16.217731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:16.218566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:16.219460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:16.219751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:16.221346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:16.229518 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:16.320677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:16.322275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47239aa0-e516-443a-8286-df960d9ed444 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:16.340901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 716/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:17.478436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3823ca-73f7-48c7-bc8c-6e1e2f5b078f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:17.479097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3823ca-73f7-48c7-bc8c-6e1e2f5b078f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:17.479097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3823ca-73f7-48c7-bc8c-6e1e2f5b078f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:17.480442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:17.486833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972437c-529e-4c6d-883c-90b36d456d54 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:17.487348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972437c-529e-4c6d-883c-90b36d456d54 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:17.487512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972437c-529e-4c6d-883c-90b36d456d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:17.563335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f972437c-529e-4c6d-883c-90b36d456d54 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:17.564554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f972437c-529e-4c6d-883c-90b36d456d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:17.631500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:17.660534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:17.810680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3823ca-73f7-48c7-bc8c-6e1e2f5b078f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:17.812151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ed3823ca-73f7-48c7-bc8c-6e1e2f5b078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:17.830496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:17.958228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3823ca-73f7-48c7-bc8c-6e1e2f5b078f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:18.044653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:18.070869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:18.071399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:18.071669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:18.071934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:18.074230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:18.166178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:18.167953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0250-6146-478a-8292-67b56d7d1ed8 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:18.186175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 717/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:19.303120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa816d1-770e-4732-ba8e-1846260efaa5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:19.303661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa816d1-770e-4732-ba8e-1846260efaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:19.303782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa816d1-770e-4732-ba8e-1846260efaa5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:19.305070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:19.311243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaf68c-60f8-48a1-8ea5-91308846175e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:19.311640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaf68c-60f8-48a1-8ea5-91308846175e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:19.311784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaf68c-60f8-48a1-8ea5-91308846175e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:19.387685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-befaf68c-60f8-48a1-8ea5-91308846175e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:19.388895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-befaf68c-60f8-48a1-8ea5-91308846175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:19.430771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:19.485608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:08:19.635916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa816d1-770e-4732-ba8e-1846260efaa5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:19.637434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7aa816d1-770e-4732-ba8e-1846260efaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:19.782668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa816d1-770e-4732-ba8e-1846260efaa5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:19.836556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:19.895445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:19.896052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:19.896331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:19.896601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:19.898186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:19.989320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:19.991011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d3f9e-9d9a-4213-bbcc-8d2734af9fd3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:20.009588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 719/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:21.140647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28410f3e-a95f-4892-aaac-388b58e52753 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:21.141215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28410f3e-a95f-4892-aaac-388b58e52753 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:21.141215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28410f3e-a95f-4892-aaac-388b58e52753 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:21.142574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:21.148866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae99b7-2691-49bc-a7de-33a3ea6277a9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:21.149314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae99b7-2691-49bc-a7de-33a3ea6277a9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:21.149423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae99b7-2691-49bc-a7de-33a3ea6277a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:21.226174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-36ae99b7-2691-49bc-a7de-33a3ea6277a9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:21.227410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-36ae99b7-2691-49bc-a7de-33a3ea6277a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:21.229412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:21.323886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:21.472141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28410f3e-a95f-4892-aaac-388b58e52753 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:21.473600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-28410f3e-a95f-4892-aaac-388b58e52753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:21.618461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28410f3e-a95f-4892-aaac-388b58e52753 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:21.647802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:21.732130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:21.732664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:21.732941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:21.733212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:21.734898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:21.822412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:21.824017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e9558-14f5-48e9-9963-e36f5cfbe8a3 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:21.842820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 720/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:23.905853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be9f8-3484-487d-ad69-b8ec834e0994 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:23.906751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be9f8-3484-487d-ad69-b8ec834e0994 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:23.906916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be9f8-3484-487d-ad69-b8ec834e0994 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:23.908279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:23.913750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10877d58-a7e8-44d3-920f-2cd58d1fdbf7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:23.914179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10877d58-a7e8-44d3-920f-2cd58d1fdbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:23.914289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10877d58-a7e8-44d3-920f-2cd58d1fdbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:23.955337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:23.979761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-10877d58-a7e8-44d3-920f-2cd58d1fdbf7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:23.981005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-10877d58-a7e8-44d3-920f-2cd58d1fdbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:24.076020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:24.359838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be9f8-3484-487d-ad69-b8ec834e0994 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:24.361365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aa9be9f8-3484-487d-ad69-b8ec834e0994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:24.463911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be9f8-3484-487d-ad69-b8ec834e0994 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:24.495835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:24.537495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:24.538096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:24.538383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:24.538686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:24.540275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:24.651614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:24.653042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c308e-7c44-47d6-a750-28d6dc331608 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:24.671276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 722/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:26.981278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d1954-cf61-4dd5-9db6-6bcaf0669161 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:26.981810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d1954-cf61-4dd5-9db6-6bcaf0669161 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:26.981810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d1954-cf61-4dd5-9db6-6bcaf0669161 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:26.983132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:26.989083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5abb37-6691-4231-9ad4-b99e1d4a32b6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:26.989537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5abb37-6691-4231-9ad4-b99e1d4a32b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:26.989645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5abb37-6691-4231-9ad4-b99e1d4a32b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:27.054926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fe5abb37-6691-4231-9ad4-b99e1d4a32b6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:27.056132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fe5abb37-6691-4231-9ad4-b99e1d4a32b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:27.068694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:27.152453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:27.437935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d1954-cf61-4dd5-9db6-6bcaf0669161 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:27.440074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ed4d1954-cf61-4dd5-9db6-6bcaf0669161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:27.542117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d1954-cf61-4dd5-9db6-6bcaf0669161 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:27.555522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:27.615057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:27.615655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:27.615933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:27.616203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:27.617784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:27.725151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:27.726602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40d52c-581c-4063-8f1d-259f199d7e8a service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:27.748830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 723/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:30.297018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e248df-5b9d-41a3-9470-3fc0bf9275d0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:30.297588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e248df-5b9d-41a3-9470-3fc0bf9275d0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:30.297588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e248df-5b9d-41a3-9470-3fc0bf9275d0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:30.298988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:30.304185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f7cb1-90d3-4961-a45c-e51c62bf08c0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:30.304615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f7cb1-90d3-4961-a45c-e51c62bf08c0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:30.304727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f7cb1-90d3-4961-a45c-e51c62bf08c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:30.352668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:30.370218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1b8f7cb1-90d3-4961-a45c-e51c62bf08c0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:30.371439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1b8f7cb1-90d3-4961-a45c-e51c62bf08c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:30.466035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:30.751618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e248df-5b9d-41a3-9470-3fc0bf9275d0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:30.753151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61e248df-5b9d-41a3-9470-3fc0bf9275d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:30.856079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e248df-5b9d-41a3-9470-3fc0bf9275d0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:30.899753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:30.933394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:30.934000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:30.934282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:30.934580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:30.936180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:31.045821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:31.047210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee16ba7-4b8d-4097-bfa5-4a88f2b5a356 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:31.066090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 725/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:33.429107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1491c-e89b-4944-afaf-0679109a3c8a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:33.429640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1491c-e89b-4944-afaf-0679109a3c8a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:33.429640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1491c-e89b-4944-afaf-0679109a3c8a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:33.431047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:33.436289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e2ec5-73a5-40c2-b6a7-171783467977 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:33.436828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e2ec5-73a5-40c2-b6a7-171783467977 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:33.437108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e2ec5-73a5-40c2-b6a7-171783467977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:33.503829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e70e2ec5-73a5-40c2-b6a7-171783467977 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:33.504973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e70e2ec5-73a5-40c2-b6a7-171783467977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:33.525918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:33.601363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:33.889004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1491c-e89b-4944-afaf-0679109a3c8a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:33.890539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4eb1491c-e89b-4944-afaf-0679109a3c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:33.996776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1491c-e89b-4944-afaf-0679109a3c8a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:34.017707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:34.071331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:34.071968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:34.072248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:34.072522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:34.074121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:34.183570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:34.185002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a85a2-06a5-4c83-9e71-f2bd28188aab service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:34.202690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 726/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:36.418198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45905c1-6383-4ef1-b6c2-6757964d45ad None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:36.418755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45905c1-6383-4ef1-b6c2-6757964d45ad None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:36.418755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45905c1-6383-4ef1-b6c2-6757964d45ad None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:36.420139 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:36.425510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b461d7-ba31-4296-aca8-7b3b5f62bedc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:36.425981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b461d7-ba31-4296-aca8-7b3b5f62bedc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:36.426082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b461d7-ba31-4296-aca8-7b3b5f62bedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:36.489111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:36.491746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b3b461d7-ba31-4296-aca8-7b3b5f62bedc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:36.492969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b3b461d7-ba31-4296-aca8-7b3b5f62bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:36.589506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:36.873920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45905c1-6383-4ef1-b6c2-6757964d45ad None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:36.875476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c45905c1-6383-4ef1-b6c2-6757964d45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:36.977513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45905c1-6383-4ef1-b6c2-6757964d45ad None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:37.025800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:37.050432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:37.051068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:37.051345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:37.051619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:37.053199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:37.164070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:37.165394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4701cd-7d4c-4c62-989b-b5e341565688 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:37.183756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 728/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:39.370135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1789ba-73c7-4f81-9f8d-c6592f3a58f6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:39.370693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1789ba-73c7-4f81-9f8d-c6592f3a58f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:39.370693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1789ba-73c7-4f81-9f8d-c6592f3a58f6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:39.372067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:39.378066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e71f88-2cfa-4be7-8fbd-ff62908846f4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:39.378480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e71f88-2cfa-4be7-8fbd-ff62908846f4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:39.378654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e71f88-2cfa-4be7-8fbd-ff62908846f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:39.446499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-28e71f88-2cfa-4be7-8fbd-ff62908846f4 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:39.447736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28e71f88-2cfa-4be7-8fbd-ff62908846f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:39.476690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:39.544446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:39.828829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1789ba-73c7-4f81-9f8d-c6592f3a58f6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:39.830383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9a1789ba-73c7-4f81-9f8d-c6592f3a58f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:39.933184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1789ba-73c7-4f81-9f8d-c6592f3a58f6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:39.961341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:40.005269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:40.005865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:40.006143 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:40.006415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:40.008029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:40.115802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:40.117238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef5bea-1f66-40ed-9f73-896a5ca0aa48 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:40.135333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 729/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:42.327387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8221-c568-4fcd-88e4-34e5bb843b07 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:42.327948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8221-c568-4fcd-88e4-34e5bb843b07 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:42.327948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8221-c568-4fcd-88e4-34e5bb843b07 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:42.329784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:42.335202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef62e8-be6a-4d30-b4aa-ce9f3626d770 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:42.335687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef62e8-be6a-4d30-b4aa-ce9f3626d770 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:42.335814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef62e8-be6a-4d30-b4aa-ce9f3626d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:42.410765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-69ef62e8-be6a-4d30-b4aa-ce9f3626d770 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:42.412248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-69ef62e8-be6a-4d30-b4aa-ce9f3626d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:42.420572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:42.508906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:42.793150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf8221-c568-4fcd-88e4-34e5bb843b07 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:42.794707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e3cf8221-c568-4fcd-88e4-34e5bb843b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:42.898169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf8221-c568-4fcd-88e4-34e5bb843b07 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:42.954965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:42.975045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:42.975662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:42.975945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:42.976219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:42.977829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:43.089448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:43.090892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3ca0b-3778-4c59-ab36-3f0371e5153d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:43.110057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 731/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:45.292443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd750a2-991b-4642-bfa5-ddd58c901000 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:45.292977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd750a2-991b-4642-bfa5-ddd58c901000 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:45.292977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd750a2-991b-4642-bfa5-ddd58c901000 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:45.294334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:45.300086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8801fa2-f2d5-44b7-bb7c-4eccafb4cc35 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:45.300512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8801fa2-f2d5-44b7-bb7c-4eccafb4cc35 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:45.300610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8801fa2-f2d5-44b7-bb7c-4eccafb4cc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:45.365091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c8801fa2-f2d5-44b7-bb7c-4eccafb4cc35 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:45.366285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c8801fa2-f2d5-44b7-bb7c-4eccafb4cc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:45.404808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:45.462859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:08:45.748297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd750a2-991b-4642-bfa5-ddd58c901000 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:45.750306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1fd750a2-991b-4642-bfa5-ddd58c901000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:45.852718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd750a2-991b-4642-bfa5-ddd58c901000 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:45.889207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:45.925478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:45.926071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:45.926350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:45.926639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:45.928200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:46.034172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:46.035556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f98bc-1133-42ff-9082-7402bd11b5d7 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:46.056156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 732/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:48.245294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32900ec8-9895-4ba3-b3c7-6cff0d0a52b1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:48.245787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32900ec8-9895-4ba3-b3c7-6cff0d0a52b1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:48.245885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32900ec8-9895-4ba3-b3c7-6cff0d0a52b1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:48.247229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:48.253203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5850c-20a3-4cd5-a357-2cbebc0cc2fc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:48.253639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5850c-20a3-4cd5-a357-2cbebc0cc2fc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:48.253750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5850c-20a3-4cd5-a357-2cbebc0cc2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:48.320810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-71e5850c-20a3-4cd5-a357-2cbebc0cc2fc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:48.321996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-71e5850c-20a3-4cd5-a357-2cbebc0cc2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:48.339983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:48.416345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:48.724020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32900ec8-9895-4ba3-b3c7-6cff0d0a52b1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:48.725004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-32900ec8-9895-4ba3-b3c7-6cff0d0a52b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:48.829496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32900ec8-9895-4ba3-b3c7-6cff0d0a52b1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:48.897478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:48.903975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:48.904582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:48.904863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:48.905131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:48.907377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:49.015827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:49.017172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7f771-3872-4816-aab3-a764cf46b03e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:49.035969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 734/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:08:51.216966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6139-853e-419e-b8ff-61161376aef7 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:51.217560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6139-853e-419e-b8ff-61161376aef7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:51.217560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6139-853e-419e-b8ff-61161376aef7 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:51.219487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:08:51.224910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59749c-1e66-44c8-937f-4bec511750cf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:51.225339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59749c-1e66-44c8-937f-4bec511750cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:51.225471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59749c-1e66-44c8-937f-4bec511750cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:51.292102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aa59749c-1e66-44c8-937f-4bec511750cf None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:08:51.293318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aa59749c-1e66-44c8-937f-4bec511750cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:51.340201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:51.387701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:08:51.670829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6139-853e-419e-b8ff-61161376aef7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:08:51.672379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-451f6139-853e-419e-b8ff-61161376aef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:51.774824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6139-853e-419e-b8ff-61161376aef7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:08:51.822373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:08:51.847914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:08:51.848513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:08:51.848825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:08:51.849104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:08:51.851421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:08:51.960288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:08:51.961652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bbda5-8b59-4d9e-b800-7572c19a64af service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:08:51.980280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 735/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:01.386135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=22 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:01.977819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=66 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:11.939533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67de1c-65c7-4767-b297-a7d5172df56d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:11.940086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67de1c-65c7-4767-b297-a7d5172df56d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:11.940086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67de1c-65c7-4767-b297-a7d5172df56d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:11.941354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 736/1471] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:11.946944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d5ec3-b053-4fe3-815c-54ece7822eba None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:11.947366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d5ec3-b053-4fe3-815c-54ece7822eba None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:11.947485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d5ec3-b053-4fe3-815c-54ece7822eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:12.015473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7e9d5ec3-b053-4fe3-815c-54ece7822eba None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:12.016728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7e9d5ec3-b053-4fe3-815c-54ece7822eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:12.110977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 736/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:12.415703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc29e-adb4-43d5-acc2-5ca794e43147 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:12.415989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc29e-adb4-43d5-acc2-5ca794e43147 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:12.416265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc29e-adb4-43d5-acc2-5ca794e43147 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:12.417588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 737/1473] 10.222.0.46 () {54 vars in 961 bytes} [Wed Sep 7 19:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:12.423656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee7e5b-223a-42bc-8b13-1e3a6cfe7306 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:12.423656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee7e5b-223a-42bc-8b13-1e3a6cfe7306 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:12.423929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee7e5b-223a-42bc-8b13-1e3a6cfe7306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:12.499559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-caee7e5b-223a-42bc-8b13-1e3a6cfe7306 None None] MFA Rules not processed for user `2f6d9e85ceed45e186421315737f9e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:12.500698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-caee7e5b-223a-42bc-8b13-1e3a6cfe7306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:12.521072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:12.576216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 737/1474] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Sep 7 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:12.584051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc29e-adb4-43d5-acc2-5ca794e43147 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:12.585586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d86dc29e-adb4-43d5-acc2-5ca794e43147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:12.689338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc29e-adb4-43d5-acc2-5ca794e43147 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:12.762118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:12.762686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:12.762981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:12.763246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:12.764805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:12.818010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:12.874121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:12.875592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f21e28-2b0c-4e9d-a9c2-99098aca53f5 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:12.894643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 738/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:15.133199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed278f2a-6ed5-4852-ac91-548c3ff396e7 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:15.133757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed278f2a-6ed5-4852-ac91-548c3ff396e7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:15.133757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed278f2a-6ed5-4852-ac91-548c3ff396e7 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:15.135128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 738/1476] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:15.141133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1b52-eadd-44a6-be82-b8a9ddb5c35e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:15.141513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1b52-eadd-44a6-be82-b8a9ddb5c35e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:15.141691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1b52-eadd-44a6-be82-b8a9ddb5c35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:15.206922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b56c1b52-eadd-44a6-be82-b8a9ddb5c35e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:15.208133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b56c1b52-eadd-44a6-be82-b8a9ddb5c35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:15.304532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 739/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:15.573888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed278f2a-6ed5-4852-ac91-548c3ff396e7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:15.575433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ed278f2a-6ed5-4852-ac91-548c3ff396e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:15.697863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed278f2a-6ed5-4852-ac91-548c3ff396e7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:15.748375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:15.791499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:15.792100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:15.792378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:15.792653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:15.793281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:15.795086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:09:15.795086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:09:15.795086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:09:15.796803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abfcaa-df04-44e1-b443-e253252628ab admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:15.815622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 739/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Sep 7 19:09:15 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:09:18.005991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7fac-d03c-42fe-8593-f3f76a59050b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:18.006591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7fac-d03c-42fe-8593-f3f76a59050b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:18.006591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7fac-d03c-42fe-8593-f3f76a59050b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:18.007908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 740/1479] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:18.013150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60b60f-bb09-46bb-b63d-ca2f84c8da4c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:18.013602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60b60f-bb09-46bb-b63d-ca2f84c8da4c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:18.013700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60b60f-bb09-46bb-b63d-ca2f84c8da4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:18.081218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ff60b60f-bb09-46bb-b63d-ca2f84c8da4c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:18.082439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ff60b60f-bb09-46bb-b63d-ca2f84c8da4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:18.181947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 740/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:18.476204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7fac-d03c-42fe-8593-f3f76a59050b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:18.477698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0c4f7fac-d03c-42fe-8593-f3f76a59050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:18.488096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:18.624532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7fac-d03c-42fe-8593-f3f76a59050b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:18.704236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:18.737871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:18.738434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:18.738733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:18.739008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:18.741005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:18.840466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:18.842578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4ebcc-eb70-41eb-b2ff-40bca376bfcd service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:18.860990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 741/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:21.185421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f652cf-0a61-46f0-a290-e1f6153f2649 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:21.185998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f652cf-0a61-46f0-a290-e1f6153f2649 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:21.185998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f652cf-0a61-46f0-a290-e1f6153f2649 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:21.187332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 741/1482] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:21.192608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da3e8e-72ed-4ff2-825b-b3f3828d961d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:21.193013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da3e8e-72ed-4ff2-825b-b3f3828d961d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:21.193146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da3e8e-72ed-4ff2-825b-b3f3828d961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:21.258790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d6da3e8e-72ed-4ff2-825b-b3f3828d961d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:21.259992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6da3e8e-72ed-4ff2-825b-b3f3828d961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:21.305963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:21.355787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 742/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:21.649320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f652cf-0a61-46f0-a290-e1f6153f2649 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:21.650866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-31f652cf-0a61-46f0-a290-e1f6153f2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:21.674714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:21.753989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f652cf-0a61-46f0-a290-e1f6153f2649 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:21.825407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:21.826002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:21.826844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:21.827113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:21.828670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:21.902361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:21.937488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:21.938964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa48e7-6c37-4c49-ade1-cbe7eecab9ba service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:21.958598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 742/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:24.159520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40591a6a-055a-49d3-bca2-d93438ab3f2c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:24.162632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40591a6a-055a-49d3-bca2-d93438ab3f2c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:24.162632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40591a6a-055a-49d3-bca2-d93438ab3f2c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:24.162632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 743/1485] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:24.167031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408a5f7-5983-4b37-b1e3-fc7489ad2601 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:24.167449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408a5f7-5983-4b37-b1e3-fc7489ad2601 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:24.167618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408a5f7-5983-4b37-b1e3-fc7489ad2601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:24.238862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4408a5f7-5983-4b37-b1e3-fc7489ad2601 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:24.240074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4408a5f7-5983-4b37-b1e3-fc7489ad2601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:24.337239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 743/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:24.631196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40591a6a-055a-49d3-bca2-d93438ab3f2c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:24.632711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40591a6a-055a-49d3-bca2-d93438ab3f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:24.736082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40591a6a-055a-49d3-bca2-d93438ab3f2c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:24.808271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:24.809737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:24.810421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:24.810728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:24.811000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:24.812641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:24.920419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:24.921780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57aa22-b84f-40fb-9b68-5cadfe2e5305 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:24.939282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 744/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:27.226064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573689-3f34-4e37-8aed-daca463d3278 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:27.226668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573689-3f34-4e37-8aed-daca463d3278 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:27.226668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573689-3f34-4e37-8aed-daca463d3278 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:27.227975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 744/1488] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:27.233706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0d888-7bb9-43f6-a1d9-31f89ae9eebe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:27.234203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0d888-7bb9-43f6-a1d9-31f89ae9eebe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:27.234306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0d888-7bb9-43f6-a1d9-31f89ae9eebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:27.300401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ee0d888-7bb9-43f6-a1d9-31f89ae9eebe None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:27.301628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ee0d888-7bb9-43f6-a1d9-31f89ae9eebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:27.356279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:27.400209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 745/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:27.695170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01573689-3f34-4e37-8aed-daca463d3278 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:27.696710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-01573689-3f34-4e37-8aed-daca463d3278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:27.732560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:27.800155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01573689-3f34-4e37-8aed-daca463d3278 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:27.872152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:27.873265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:27.873549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:27.873816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:27.875435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:27.957069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:27.985847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:27.987324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239533e-fdd4-405b-aab5-e8a4c9b44d74 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:28.006424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 745/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:30.518162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2da5-5192-4f84-b32d-212cc5cfc4fa None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:30.518736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2da5-5192-4f84-b32d-212cc5cfc4fa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:30.518736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2da5-5192-4f84-b32d-212cc5cfc4fa None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:30.519974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 746/1491] 10.222.0.46 () {54 vars in 927 bytes} [Wed Sep 7 19:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:09:30.525332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e274b0-2264-439f-a7e8-6b137c21d1fa None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:30.525743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e274b0-2264-439f-a7e8-6b137c21d1fa None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:30.525886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e274b0-2264-439f-a7e8-6b137c21d1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:30.595752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-51e274b0-2264-439f-a7e8-6b137c21d1fa None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:09:30.596981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-51e274b0-2264-439f-a7e8-6b137c21d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:30.693563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 746/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Wed Sep 7 19:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:09:30.982252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de2da5-5192-4f84-b32d-212cc5cfc4fa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:09:30.983826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-34de2da5-5192-4f84-b32d-212cc5cfc4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:31.087151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de2da5-5192-4f84-b32d-212cc5cfc4fa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:09:31.161606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:09:31.162210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:09:31.162490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:09:31.162788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:09:31.164383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:09:31.172749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:31.272599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:09:31.274060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2124d4-49c9-4681-a688-b332f8a92306 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:09:31.293712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 747/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:09:40.692030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:09:41.291457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=50 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:53.283957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:53.284480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:53.284480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:53.360109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:10:53.361321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:53.456405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 747/1494] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:10:53.464326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:53.464801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:53.464905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:53.541557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:10:53.542787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:53.636827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 748/1495] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:10:53.641359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:53.642638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:53.654019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd68fa-ab23-423b-ac2d-6110a902a02e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:53.718012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:53.751977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:53.752579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:53.752856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:53.753126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:53.753763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:10:53.754062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:53.755105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:53.755105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:10:53.755105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:10:53.756081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:53.767778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 748/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:10:53.776538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:53.785769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703a91d-5b51-4170-8640-261816a05c22 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:53.854991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:53.889060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:53.889596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:53.889876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:53.890197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:53.890931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:53.892725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:53.892725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:10:53.892725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:10:53.893916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:54.038807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 749/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:10:53 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:10:54.043602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:54.045122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:54.194357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b8925-c875-4905-9bc6-8c80b7ca573b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:54.291788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:54.292439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:54.292656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:54.292979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:54.293600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:54.294640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:54.294640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:10:54.294640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:10:54.295853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:54.394189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 749/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:10:54 2022] POST /identity/v3/users => generated 289 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:10:54.398293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:54.400514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:54.461086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68261ae-19d3-41f7-8003-7a7bacc5f16c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:54.544994 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:54.555810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:10:54.560652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:10:54.561260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:54.572918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 750/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:10:54.577079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:54.578665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:54.590157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6fae-a676-401e-b5bb-bc3843bed014 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:54.615373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:54.684456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:54.684986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:54.685258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:54.685541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:54.686157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:54.687195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:54.687195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:10:54.687195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:10:54.688070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb784b-f01c-4904-af92-cac75175f1ad admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:54.701502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 750/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:10:54.705622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:54.707152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:54.739813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58366510-4a14-4709-8dad-71e6bfc74355 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:54.833401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:54.834053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:54.834323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:54.834613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] PATH_INFO: `/v3/projects/6fa6e3d37f2d4df599bf944e1ec35620/users/bd34269b0fa340d3a1b61201be1125eb/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:54.856572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa6e3d37f2d4df599bf944e1ec35620, user_id=bd34269b0fa340d3a1b61201be1125eb, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:54.857576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:54.857576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:10:54.857576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:10:54.859525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996a7c7-e234-4e6b-ae5a-c33dba3ae085 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:54.943575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 751/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:10:54 2022] PUT /identity/v3/projects/6fa6e3d37f2d4df599bf944e1ec35620/users/bd34269b0fa340d3a1b61201be1125eb/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:10:54.955834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94241223-6f95-458a-9870-8eb0e05f7ad4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:54.956115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94241223-6f95-458a-9870-8eb0e05f7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:54.956388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94241223-6f95-458a-9870-8eb0e05f7ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:55.012985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-94241223-6f95-458a-9870-8eb0e05f7ad4 None None] MFA Rules not processed for user `bd34269b0fa340d3a1b61201be1125eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:10:55.014175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-94241223-6f95-458a-9870-8eb0e05f7ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:55.089216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 751/1502] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:10:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:10:55.103034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362c4f-4da5-4431-90fd-fdb8dd43aed6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:55.103317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362c4f-4da5-4431-90fd-fdb8dd43aed6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:55.103589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362c4f-4da5-4431-90fd-fdb8dd43aed6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:55.104917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 752/1503] 10.222.0.46 () {54 vars in 964 bytes} [Wed Sep 7 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:10:55.109780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a342b-8d64-4409-9394-d97a12c4ed39 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:55.110066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a342b-8d64-4409-9394-d97a12c4ed39 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:55.110338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a342b-8d64-4409-9394-d97a12c4ed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:55.141630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:55.187254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d69a342b-8d64-4409-9394-d97a12c4ed39 None None] MFA Rules not processed for user `885f09bf588a442e931aae0da51d0ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:10:55.188462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d69a342b-8d64-4409-9394-d97a12c4ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:55.264026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 752/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Wed Sep 7 19:10:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:10:55.271725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84362c4f-4da5-4431-90fd-fdb8dd43aed6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:55.273228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-84362c4f-4da5-4431-90fd-fdb8dd43aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:55.324111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:55.377900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84362c4f-4da5-4431-90fd-fdb8dd43aed6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:55.450796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:55.451418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:55.451694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:55.451963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:55.453540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:55.552070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:55.553446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:55.557201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:55.571621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 753/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:10:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:10:55.617128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06020b-e162-4c71-a615-0df444ff90e8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:55.617574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06020b-e162-4c71-a615-0df444ff90e8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:55.617782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06020b-e162-4c71-a615-0df444ff90e8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:55.619075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 753/1506] 10.222.0.46 () {52 vars in 830 bytes} [Wed Sep 7 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:10:55.622532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:55.623853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:55.722248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679906b2-3b08-4921-b635-0d116b1d5c68 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:55.814339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:55.815045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:55.815316 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:55.815577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] PATH_INFO: `/v3/users/bd34269b0fa340d3a1b61201be1125eb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:55.828665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd34269b0fa340d3a1b61201be1125eb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:55.829673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:55.829673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:10:55.829673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:10:55.830923 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:55.841020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:56.060390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd34269b0fa340d3a1b61201be1125eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:56.060928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd34269b0fa340d3a1b61201be1125eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:56.061351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd34269b0fa340d3a1b61201be1125eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:56.078243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd34269b0fa340d3a1b61201be1125eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:56.098436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd34269b0fa340d3a1b61201be1125eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:56.179262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 754/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:10:55 2022] DELETE /identity/v3/users/bd34269b0fa340d3a1b61201be1125eb => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:10:56.184129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:56.184421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:56.184697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:56.241172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:10:56.242389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:56.318367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:56.331790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 754/1508] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:10:56.340713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:56.341903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:56.443173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369744-1214-4403-8ae3-f8f81ff2b302 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:56.497490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:10:56.516273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:56.516879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:56.517159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:56.517431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:56.519079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:56.629116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:56.630473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64e13a-6a06-4214-a37f-8f12764254ac service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:56.648771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 755/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:10:57.153839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:10:57.160260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:10:57.166932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301c5af-6bbf-4299-9434-da6c89c7f2b6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:10:57.263429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:10:57.264055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:10:57.264328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:10:57.264595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] PATH_INFO: `/v3/projects/6fa6e3d37f2d4df599bf944e1ec35620` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:10:57.274787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa6e3d37f2d4df599bf944e1ec35620)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:10:57.275812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:10:57.275812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:10:57.275812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:10:57.277026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:10:57.397986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa6e3d37f2d4df599bf944e1ec35620', 'request_id': 'req-7482b436-8491-4383-bfc4-54d499c7e5dd'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:57.412619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa6e3d37f2d4df599bf944e1ec35620', 'request_id': 'req-7482b436-8491-4383-bfc4-54d499c7e5dd'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:57.418693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa6e3d37f2d4df599bf944e1ec35620', 'request_id': 'req-7482b436-8491-4383-bfc4-54d499c7e5dd'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:57.530871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] The token cache is being invalidate because project 6fa6e3d37f2d4df599bf944e1ec35620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:10:57.531161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:57.531589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7482b436-8491-4383-bfc4-54d499c7e5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:10:57.533023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 755/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:10:57 2022] DELETE /identity/v3/projects/6fa6e3d37f2d4df599bf944e1ec35620 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:11:06.646458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=62 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:11:07.528598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:12:28.274115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8069145-700c-4491-be7b-77102e4b8c26 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:28.274656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8069145-700c-4491-be7b-77102e4b8c26 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:28.274848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8069145-700c-4491-be7b-77102e4b8c26 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:28.276049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:28.280751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae40bc8-8d5b-44aa-9b47-8d5d6b6a888d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:28.281157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae40bc8-8d5b-44aa-9b47-8d5d6b6a888d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:28.281287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae40bc8-8d5b-44aa-9b47-8d5d6b6a888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:28.359962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5ae40bc8-8d5b-44aa-9b47-8d5d6b6a888d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:28.361161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5ae40bc8-8d5b-44aa-9b47-8d5d6b6a888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:28.464001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:28.620521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8069145-700c-4491-be7b-77102e4b8c26 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:28.622709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d8069145-700c-4491-be7b-77102e4b8c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:28.739947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8069145-700c-4491-be7b-77102e4b8c26 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:28.745420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb8ab9-29fe-41a8-9f18-9ae03bf7a264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:28.746013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8ab9-29fe-41a8-9f18-9ae03bf7a264 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:28.746290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8ab9-29fe-41a8-9f18-9ae03bf7a264 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:28.746581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8ab9-29fe-41a8-9f18-9ae03bf7a264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:28.754538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb8ab9-29fe-41a8-9f18-9ae03bf7a264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:28.756177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb8ab9-29fe-41a8-9f18-9ae03bf7a264 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:28.765396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 757/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:29.686456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1aa60-013f-42e6-a0ca-1c8634a77f84 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:29.687066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1aa60-013f-42e6-a0ca-1c8634a77f84 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:29.687066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1aa60-013f-42e6-a0ca-1c8634a77f84 None None] PATH_INFO: `/v3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:29.688294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 757/1514] 10.222.0.25 () {54 vars in 925 bytes} [Wed Sep 7 19:12:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:12:29.694639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb0b9-d297-470a-a32a-9362d7bbff49 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:29.695048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb0b9-d297-470a-a32a-9362d7bbff49 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:29.695206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb0b9-d297-470a-a32a-9362d7bbff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:29.758086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7e4fb0b9-d297-470a-a32a-9362d7bbff49 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:29.759393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7e4fb0b9-d297-470a-a32a-9362d7bbff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:29.828183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:12:29.833772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 758/1515] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Sep 7 19:12:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:29.835092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf100af-9186-49d5-b0d4-a57c4eb77316 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:29.835367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf100af-9186-49d5-b0d4-a57c4eb77316 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:29.835639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf100af-9186-49d5-b0d4-a57c4eb77316 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:29.836963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 758/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 19:12:29.843642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55f0b-26de-4db4-a2bd-8b98f456ffe8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:29.843957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55f0b-26de-4db4-a2bd-8b98f456ffe8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:29.844257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55f0b-26de-4db4-a2bd-8b98f456ffe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:29.863942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2b55f0b-26de-4db4-a2bd-8b98f456ffe8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:29.865126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2b55f0b-26de-4db4-a2bd-8b98f456ffe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:29.872967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 759/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:30.023753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf100af-9186-49d5-b0d4-a57c4eb77316 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:30.038619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf100af-9186-49d5-b0d4-a57c4eb77316 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:30.043885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19c2b4-a234-4728-b37c-65c8964f8d09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:30.044473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19c2b4-a234-4728-b37c-65c8964f8d09 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:30.044743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19c2b4-a234-4728-b37c-65c8964f8d09 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:30.045008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19c2b4-a234-4728-b37c-65c8964f8d09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:30.052711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19c2b4-a234-4728-b37c-65c8964f8d09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:30.054304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19c2b4-a234-4728-b37c-65c8964f8d09 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:30.063399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 759/1518] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:38.653392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fcd30-6e34-4bac-8281-ed9948590555 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:38.653894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fcd30-6e34-4bac-8281-ed9948590555 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:38.654004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fcd30-6e34-4bac-8281-ed9948590555 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:38.655478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 760/1519] 10.222.0.46 () {54 vars in 961 bytes} [Wed Sep 7 19:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:38.660937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ad02d-5c32-4092-9555-b19faa9ddd4d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:38.661411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ad02d-5c32-4092-9555-b19faa9ddd4d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:38.661522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ad02d-5c32-4092-9555-b19faa9ddd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:38.681692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-290ad02d-5c32-4092-9555-b19faa9ddd4d None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:38.683249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-290ad02d-5c32-4092-9555-b19faa9ddd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:38.691619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 760/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Sep 7 19:12:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:12:38.699891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fcd30-6e34-4bac-8281-ed9948590555 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:38.713255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fcd30-6e34-4bac-8281-ed9948590555 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:38.718178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877ed53-2550-4bae-9c57-9a3c0d2c783b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:38.718788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877ed53-2550-4bae-9c57-9a3c0d2c783b service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:38.719065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877ed53-2550-4bae-9c57-9a3c0d2c783b service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:38.719334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877ed53-2550-4bae-9c57-9a3c0d2c783b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:38.727430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877ed53-2550-4bae-9c57-9a3c0d2c783b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:38.728990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877ed53-2550-4bae-9c57-9a3c0d2c783b service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:38.738460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 761/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:41.537319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c569d6d-b910-45e0-a6cb-cfaab61adc85 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:41.537852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c569d6d-b910-45e0-a6cb-cfaab61adc85 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:41.537852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c569d6d-b910-45e0-a6cb-cfaab61adc85 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:41.539252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:41.545536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186fd1-3d2d-4aa0-9f9b-975a7915975c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:41.545958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186fd1-3d2d-4aa0-9f9b-975a7915975c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:41.546082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186fd1-3d2d-4aa0-9f9b-975a7915975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:41.568832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-46186fd1-3d2d-4aa0-9f9b-975a7915975c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:41.570050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-46186fd1-3d2d-4aa0-9f9b-975a7915975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:41.578419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:41.733674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c569d6d-b910-45e0-a6cb-cfaab61adc85 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:41.748421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c569d6d-b910-45e0-a6cb-cfaab61adc85 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:41.753894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7490af-6265-4f1d-ac83-13a8c449323a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:41.754492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7490af-6265-4f1d-ac83-13a8c449323a service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:41.754792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7490af-6265-4f1d-ac83-13a8c449323a service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:41.755067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7490af-6265-4f1d-ac83-13a8c449323a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:41.763047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7490af-6265-4f1d-ac83-13a8c449323a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:41.764617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7490af-6265-4f1d-ac83-13a8c449323a service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:41.774532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 762/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:42.843080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc6a4-5088-4e0f-8523-013147cf07f2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:42.843718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc6a4-5088-4e0f-8523-013147cf07f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:42.843718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc6a4-5088-4e0f-8523-013147cf07f2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:42.844995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 19:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:42.851634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ace13-0a50-40d0-8710-94cdd008cffb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:42.852049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ace13-0a50-40d0-8710-94cdd008cffb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:42.852338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ace13-0a50-40d0-8710-94cdd008cffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:42.872188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-430ace13-0a50-40d0-8710-94cdd008cffb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:42.873397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-430ace13-0a50-40d0-8710-94cdd008cffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:42.881438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:43.034681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fc6a4-5088-4e0f-8523-013147cf07f2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:43.049874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fc6a4-5088-4e0f-8523-013147cf07f2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:43.056088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff406975-fced-46ae-9efc-8cb683c2a9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:43.056723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff406975-fced-46ae-9efc-8cb683c2a9c8 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:43.057002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff406975-fced-46ae-9efc-8cb683c2a9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:43.057275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff406975-fced-46ae-9efc-8cb683c2a9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:43.067583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff406975-fced-46ae-9efc-8cb683c2a9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:43.067583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff406975-fced-46ae-9efc-8cb683c2a9c8 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:43.076816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 764/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:47.149484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068b954-486b-452f-9eb1-e952a2cdad48 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:47.150008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068b954-486b-452f-9eb1-e952a2cdad48 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:47.150008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068b954-486b-452f-9eb1-e952a2cdad48 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:47.151367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:47.156532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb69f8-b5dd-4db0-8eaf-f4cff19dbf74 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:47.157217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb69f8-b5dd-4db0-8eaf-f4cff19dbf74 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:47.157343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb69f8-b5dd-4db0-8eaf-f4cff19dbf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:47.177078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-95eb69f8-b5dd-4db0-8eaf-f4cff19dbf74 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:47.178784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95eb69f8-b5dd-4db0-8eaf-f4cff19dbf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:47.186282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:47.489967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068b954-486b-452f-9eb1-e952a2cdad48 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:47.491693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2068b954-486b-452f-9eb1-e952a2cdad48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:47.523975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:12:47.569711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068b954-486b-452f-9eb1-e952a2cdad48 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:47.573460 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d93e9-0878-4d83-bda8-76e214de34e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:47.574075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d93e9-0878-4d83-bda8-76e214de34e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:47.574341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d93e9-0878-4d83-bda8-76e214de34e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:47.574650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d93e9-0878-4d83-bda8-76e214de34e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:47.583478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d93e9-0878-4d83-bda8-76e214de34e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:47.584901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d93e9-0878-4d83-bda8-76e214de34e7 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:47.594304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 765/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:49.719197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f09a94-624a-4543-b483-1d423b75623d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:49.719773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f09a94-624a-4543-b483-1d423b75623d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:49.719773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f09a94-624a-4543-b483-1d423b75623d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:49.721068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:49.726257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0c035-45b1-46e7-82c8-9f5f998a60fa None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:49.726747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0c035-45b1-46e7-82c8-9f5f998a60fa None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:49.726855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0c035-45b1-46e7-82c8-9f5f998a60fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:49.745648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cff0c035-45b1-46e7-82c8-9f5f998a60fa None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:49.746872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cff0c035-45b1-46e7-82c8-9f5f998a60fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:49.755163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:50.042789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f09a94-624a-4543-b483-1d423b75623d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:50.053854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f09a94-624a-4543-b483-1d423b75623d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:50.057393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7a5db-6374-4d79-a6fb-10e27ea06169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:50.057971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7a5db-6374-4d79-a6fb-10e27ea06169 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:50.058243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7a5db-6374-4d79-a6fb-10e27ea06169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:50.058536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7a5db-6374-4d79-a6fb-10e27ea06169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:50.067825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7a5db-6374-4d79-a6fb-10e27ea06169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:50.069216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7a5db-6374-4d79-a6fb-10e27ea06169 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:50.078498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 767/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:53.862668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6984da-2655-48e4-966c-b44c97f1f68e None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:53.863263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6984da-2655-48e4-966c-b44c97f1f68e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:53.863263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6984da-2655-48e4-966c-b44c97f1f68e None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:53.864608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:53.869950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc8c70-feb5-4055-b4ca-2a4896f037b9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:53.870421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc8c70-feb5-4055-b4ca-2a4896f037b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:53.870538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc8c70-feb5-4055-b4ca-2a4896f037b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:53.889461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-61bc8c70-feb5-4055-b4ca-2a4896f037b9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:53.890670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61bc8c70-feb5-4055-b4ca-2a4896f037b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:53.899236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:54.208913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6984da-2655-48e4-966c-b44c97f1f68e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:54.210643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0c6984da-2655-48e4-966c-b44c97f1f68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:54.297703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6984da-2655-48e4-966c-b44c97f1f68e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:54.301346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa11b97-2ed4-4e6b-932f-dc7c1b8914a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:54.301957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa11b97-2ed4-4e6b-932f-dc7c1b8914a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:54.302236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa11b97-2ed4-4e6b-932f-dc7c1b8914a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:54.303126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa11b97-2ed4-4e6b-932f-dc7c1b8914a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:54.311748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa11b97-2ed4-4e6b-932f-dc7c1b8914a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:54.313172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa11b97-2ed4-4e6b-932f-dc7c1b8914a0 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:54.323288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 768/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:12:56.473373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7a4d9-165b-4674-8836-6a470fa97dda None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:56.473926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7a4d9-165b-4674-8836-6a470fa97dda None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:56.473926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7a4d9-165b-4674-8836-6a470fa97dda None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:56.475285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 19:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:12:56.480619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1d73-9830-4238-ba08-33374ed75f88 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:56.481041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1d73-9830-4238-ba08-33374ed75f88 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:56.481176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1d73-9830-4238-ba08-33374ed75f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:56.501087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d74a1d73-9830-4238-ba08-33374ed75f88 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:12:56.502216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d74a1d73-9830-4238-ba08-33374ed75f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:56.510296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:12:56.819634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7a4d9-165b-4674-8836-6a470fa97dda None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:12:56.821314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4aa7a4d9-165b-4674-8836-6a470fa97dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:12:56.834722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7a4d9-165b-4674-8836-6a470fa97dda None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:12:56.838156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71457391-ac4f-4cd7-b3db-df8a88414089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:12:56.838771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71457391-ac4f-4cd7-b3db-df8a88414089 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:12:56.839049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71457391-ac4f-4cd7-b3db-df8a88414089 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:12:56.839320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71457391-ac4f-4cd7-b3db-df8a88414089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:12:56.848642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71457391-ac4f-4cd7-b3db-df8a88414089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:12:56.850027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71457391-ac4f-4cd7-b3db-df8a88414089 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:12:56.859430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 770/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:13:06.496196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=63 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:13:06.855207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=37 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:21.736202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53eb4-9740-425e-98d3-d057eb6c2b8d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:21.737420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53eb4-9740-425e-98d3-d057eb6c2b8d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:21.737895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53eb4-9740-425e-98d3-d057eb6c2b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:21.765021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ece53eb4-9740-425e-98d3-d057eb6c2b8d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:21.767076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ece53eb4-9740-425e-98d3-d057eb6c2b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:21.782262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 770/1540] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:21.804332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:21.805360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:21.805872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:21.829265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbe09f-40a4-4740-8ee4-ea8cc296fe3b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:21.829909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbe09f-40a4-4740-8ee4-ea8cc296fe3b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:21.830409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbe09f-40a4-4740-8ee4-ea8cc296fe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:21.860974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:21.864308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:21.870497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c1cbe09f-40a4-4740-8ee4-ea8cc296fe3b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:21.872746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c1cbe09f-40a4-4740-8ee4-ea8cc296fe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:21.888660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 771/1541] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:21.892935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:21.900258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 771/1542] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:21.906209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee2b81-7c85-4081-a84b-86aa3fce7b52 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:21.910047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af041432-5d0d-49bf-b02c-73beeb0c00b9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:21.910716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af041432-5d0d-49bf-b02c-73beeb0c00b9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:21.911227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af041432-5d0d-49bf-b02c-73beeb0c00b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:21.924355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:21.930570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:21.930570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:21.930570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:21.930570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:21.930570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:21.931089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:14:21.931089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:14:21.931089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:14:21.931089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:21.942029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-af041432-5d0d-49bf-b02c-73beeb0c00b9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:21.947994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-af041432-5d0d-49bf-b02c-73beeb0c00b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:21.961636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 772/1543] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:14:21.966923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:21.966923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:21.966923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:21.967238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 772/1544] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:21.974337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:21.998211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ea45-2c3c-4173-9869-bc7c2709cb75 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.005707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.013730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.013884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.013884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.013884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.013884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.014231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:22.014231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.014542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1a1e7-5b67-4eb3-b7cd-f2e9fb50ca23 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.040592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 773/1545] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.042715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.054350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 773/1546] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:14:22.056005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6784632-bec8-4b1c-9e3b-3ed7a8f74c78 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.056117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6784632-bec8-4b1c-9e3b-3ed7a8f74c78 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.058598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6784632-bec8-4b1c-9e3b-3ed7a8f74c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.058909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d678e-0bfa-4415-beea-a5c46e359ea5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.065375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.066244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.066746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.067289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.068107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.069737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:14:22.069737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:14:22.069737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:14:22.071342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.089372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f6784632-bec8-4b1c-9e3b-3ed7a8f74c78 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.090750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6784632-bec8-4b1c-9e3b-3ed7a8f74c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.106622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 774/1547] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.108477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb165-fde3-4329-b91f-b19a34800d47 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.108790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb165-fde3-4329-b91f-b19a34800d47 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.109097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb165-fde3-4329-b91f-b19a34800d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.129128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c3acb165-fde3-4329-b91f-b19a34800d47 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.130374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c3acb165-fde3-4329-b91f-b19a34800d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.138258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 775/1548] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.139457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbea0-922a-488c-ad4f-3e1cc4fe4594 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.139828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbea0-922a-488c-ad4f-3e1cc4fe4594 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.140657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbea0-922a-488c-ad4f-3e1cc4fe4594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.160430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-59cfbea0-922a-488c-ad4f-3e1cc4fe4594 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.161634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59cfbea0-922a-488c-ad4f-3e1cc4fe4594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.174577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 776/1549] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.174577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765aff0-7b2d-49ab-8710-eff49137c234 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.174577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765aff0-7b2d-49ab-8710-eff49137c234 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.174577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765aff0-7b2d-49ab-8710-eff49137c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.189378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 774/1550] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.190042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.198854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2765aff0-7b2d-49ab-8710-eff49137c234 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.200178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2765aff0-7b2d-49ab-8710-eff49137c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.255657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bd4da-6925-4002-8396-4046421df96c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.260255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cf8ec-f021-4933-9fd8-3795040aef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.260874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cf8ec-f021-4933-9fd8-3795040aef43 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.261152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cf8ec-f021-4933-9fd8-3795040aef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.261426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cf8ec-f021-4933-9fd8-3795040aef43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.261577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 777/1551] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.262053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cf8ec-f021-4933-9fd8-3795040aef43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.262798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e4438-2266-443f-9f7d-414a70c263f3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.263083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e4438-2266-443f-9f7d-414a70c263f3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.263354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e4438-2266-443f-9f7d-414a70c263f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.263988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cf8ec-f021-4933-9fd8-3795040aef43 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.283134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0e6e4438-2266-443f-9f7d-414a70c263f3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.284340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0e6e4438-2266-443f-9f7d-414a70c263f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.292296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 778/1552] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.293457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bab4c-17cb-4fd9-ae36-7996dd81dc30 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.296709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bab4c-17cb-4fd9-ae36-7996dd81dc30 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.296709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bab4c-17cb-4fd9-ae36-7996dd81dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.314747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f68bab4c-17cb-4fd9-ae36-7996dd81dc30 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.315938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f68bab4c-17cb-4fd9-ae36-7996dd81dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.323880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 779/1553] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.325459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70472c2-56ed-4e3f-8385-c8e1dc286594 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.325459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70472c2-56ed-4e3f-8385-c8e1dc286594 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.325656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70472c2-56ed-4e3f-8385-c8e1dc286594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.345092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a70472c2-56ed-4e3f-8385-c8e1dc286594 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.346271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a70472c2-56ed-4e3f-8385-c8e1dc286594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.354091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 780/1554] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.357408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.357408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.357408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.375520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.376698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.384877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 775/1555] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.390522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 781/1556] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.390672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.390897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.391001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.394980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2708be5-3515-41c6-b66d-1053e572713f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.408556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.409751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.453289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.453900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.454180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.454454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.455121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.456126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:14:22.456126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:14:22.456126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:14:22.457340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.469792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 776/1557] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.470809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.480249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afe2c6-8868-489e-a6a1-273881951261 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.486171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.486798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.487076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.487343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.487977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:22.488274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.489992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778e6e-70d3-40fb-92f8-b7c1f3522f58 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.501990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 777/1558] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:22.506180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.506180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.506180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.522532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.523914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.524330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 782/1559] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.525038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.535572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 778/1560] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.535977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595d572-8835-4fcb-b003-0a6e390064e1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.536407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.540979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.541604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.541883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.542150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.542823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:22.543124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.544812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.546065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168f65c-d08a-47ad-9b2b-f667b62de4a4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.551380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.551973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.552593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.552733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:22.552827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.553250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:22.553663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.555311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.557750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:22.558705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 783/1561] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:22.559761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.569694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 779/1562] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:22.569694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.570018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89eece-75cd-4da1-a984-66103d2600f1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.574488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.575115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.575393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.576033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.576684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:22.576987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.578591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3e65e-167d-4547-bfb1-da656fbda84b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.581396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0050f-9b01-4504-b8e1-3c9e4ea3b2f9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.586346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.586969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.587247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.587521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.588137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.590045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.598448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 784/1563] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:22.598448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.598448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.598448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.615022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:22.616354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:22.624195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 785/1564] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:22.624931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.633957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea647e4-e055-42c0-a0e0-2f5416d197dc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.638460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.639021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.639296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.639566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.640207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.642081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.689700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 780/1565] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.690542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.700148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7a652-52c0-4db9-b5c6-3b26c5a69f96 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.704753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.705314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.705587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.705850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.706464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.708068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.717764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 781/1566] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:22.719239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.728829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb7322-a007-4e35-876f-cc2c3d8444c6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.733325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.736275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.736888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.736888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.737415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:22.737762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.739344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.752101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 782/1567] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:22.754466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.757886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 786/1568] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/projects => generated 393 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.758884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.762338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa501d-b800-487e-93fc-ac940c710d61 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.818710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.819324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.819601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.819876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.820508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.822220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d942-5e37-4a68-b7e6-d0a88544f1b2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:22.822862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.826955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:22.827583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:22.827853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:22.828117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:22.828787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:22.830672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:22.954507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 783/1569] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.955489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:22.971342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 787/1570] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:22.971342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.022582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce136d9-4bc8-475e-8ad7-3a6039b0aab9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.027317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.028019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.028293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.028557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.029433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.031931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.032065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe347e-4d3b-4e38-b32e-003bb2cb8ded admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.036585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.037186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.037474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.037747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.038392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:23.038731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.040317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.052604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 788/1571] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:23.059296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.062856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede71f5e-45c6-4a85-94c7-6ba08ef3c8cc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.067668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.068330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.068606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.068872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.069638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.071258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.081631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 789/1572] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:23.082405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.091501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a273-f562-47d7-b89a-cb57ced8b4e5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.096525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.097061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.097334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.097602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.098197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.099795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.109387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 790/1573] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:23.110123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.119095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8bcc9-6e32-4447-970e-bc40ba1b426b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.123866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.124428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.124707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.124984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.125628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.127563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.138355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:23.150921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 784/1574] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:22 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.150921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.176352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:23.209720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292446fc-6063-4878-abc8-dca6e2ccf3f8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.214410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.215039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.215311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.215579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.216207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.218721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.234905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 791/1575] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.234905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.242245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b448c-0568-4252-9ab5-ff88138da02e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.247342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.247595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.247815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.248089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.248721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.251182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.346816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 792/1576] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.347970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.348094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 785/1577] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.348997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.358421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a8b8a-dcc3-4a3a-82a0-666c3ec55c3c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.411970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6845-6a7d-4303-b57f-d7e4f71dff5c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.414114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.414760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.415103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.415316 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.415952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.416927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.417526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.417804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.417961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.418077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.418733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.420635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.527238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 793/1578] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/users => generated 282 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.528211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.539243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc9a23-7076-48da-b516-e493e7f50b8a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.541291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 786/1579] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.541291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.599497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.600168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.600449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.600723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.601366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.601756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e478a2-d5f9-4c80-a841-8a7c81b81429 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.602995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.607712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.608366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.608646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.608923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] PATH_INFO: `/v3/projects/d3e10ea750ae4455b7bc482b0db2ce6d/users/ddf07a7d8aed4b3faf058c66347cf315/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.613149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 794/1580] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:23.613934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.623530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd23a-7c84-4cee-abcc-44246afa7cb0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.624576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e10ea750ae4455b7bc482b0db2ce6d, user_id=ddf07a7d8aed4b3faf058c66347cf315, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.626342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:14:23.626342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:14:23.626342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:14:23.628291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.629264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.629927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.630276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.630662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.631356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.633295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.698732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 787/1581] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:23 2022] PUT /identity/v3/projects/d3e10ea750ae4455b7bc482b0db2ce6d/users/ddf07a7d8aed4b3faf058c66347cf315/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:23.699556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.718275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:23.727414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 795/1582] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.731910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.738540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5630cb-2307-40b8-8322-e6d27c711e75 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.763125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c798b-0f18-474d-b6b8-1e7460e23949 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.768090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.768715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.769032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.769304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.769971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.771610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.782486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 788/1583] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:23.783652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.787541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:23.794089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73bbaa-3020-463d-9898-af0872dd231a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.799448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.800071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.800355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.800534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.800649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.801115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.801307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.801415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.801658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.802333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.803667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.803977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.818750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 796/1584] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:23.818750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.828877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62666ee1-c617-4fe6-96cd-794ecda268c0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.833725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.834340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.834650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.834926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.835562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.837128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.848299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 797/1585] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:23.851527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.858195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650d3a4-26cd-41bd-8d6c-01035487fcac admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.863275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c70f1-7f6d-46ab-8e67-107796ddec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.863827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c70f1-7f6d-46ab-8e67-107796ddec31 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.864103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c70f1-7f6d-46ab-8e67-107796ddec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.864385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c70f1-7f6d-46ab-8e67-107796ddec31 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.864988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c70f1-7f6d-46ab-8e67-107796ddec31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.866860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c70f1-7f6d-46ab-8e67-107796ddec31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.877507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 789/1586] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.880322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:23.888295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4dc8d-8630-45a5-b703-907d35eb4145 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:23.892971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:23.893590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.893866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.894179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] PATH_INFO: `/v3/projects/8a7d9bc5395b4fa09a1024d20a980991/users/920bdd998f0c4b62b108afc799a43964/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:23.909318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7d9bc5395b4fa09a1024d20a980991, user_id=920bdd998f0c4b62b108afc799a43964, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:23.912123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:23.990766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 798/1587] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:23.992484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:23.992774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:23.993109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.011687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:24.013120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:24.019029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 790/1588] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:23 2022] PUT /identity/v3/projects/8a7d9bc5395b4fa09a1024d20a980991/users/920bdd998f0c4b62b108afc799a43964/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:24.023543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.078873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 799/1589] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:24.080049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.084500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514d4e8-703c-4241-8395-5fd2548187cb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.090011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.090189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5267ae1-c52a-4b12-bfc1-ff3969c90d6f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.090663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.090942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.091214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.091919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.093478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.095434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.096033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.096318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.096588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.097255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.098858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.104341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 791/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.105032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.115523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 800/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.115523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.115780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38988b9-ebb4-4960-9d4f-d8fe6ef475ef admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.119943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.120520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.120796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.121068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.121692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.123073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869cbf-1d86-42bf-85f8-456cf91c99f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.123299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.127808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.128393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.128668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.128940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.129568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.131188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.133739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 792/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.134667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.145406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 801/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.145406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.145671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd894de-5863-40b0-9b4a-a76e45b2fa98 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.149162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99f488-75c2-4f94-884b-15162bfae716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.150334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99f488-75c2-4f94-884b-15162bfae716 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.150646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99f488-75c2-4f94-884b-15162bfae716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.151085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99f488-75c2-4f94-884b-15162bfae716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.151725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99f488-75c2-4f94-884b-15162bfae716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.153085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbc690-93d1-49fd-9519-2332ac3776cb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.153295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99f488-75c2-4f94-884b-15162bfae716 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.158793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.159374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.159653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.159926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.160556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.162111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.165679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 793/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.167676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56543da5-2341-4d31-99da-138530b1961b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.167676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56543da5-2341-4d31-99da-138530b1961b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.167676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56543da5-2341-4d31-99da-138530b1961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.176710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 802/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.178370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.187827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-56543da5-2341-4d31-99da-138530b1961b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:24.189456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-56543da5-2341-4d31-99da-138530b1961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:24.196902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 794/1596] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:24.197750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56543da5-2341-4d31-99da-138530b1961b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.214645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56543da5-2341-4d31-99da-138530b1961b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.219780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.220391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.223082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.223082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.223082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.223424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.233811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 795/1597] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.235579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.245728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458353bf-80e3-42cc-bea2-dc441be5e6f9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.250625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.251276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.251557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.251958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] PATH_INFO: `/v3/projects/f0ca8c7ab1e64a0bb02a4941fe7b8105/users/847f6bef4e7b44f4b0adf1bda9dadb7e/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.257754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffc98c-c791-4591-b384-38004bcc892f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.261442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.262057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.262454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.262666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.269844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ca8c7ab1e64a0bb02a4941fe7b8105, user_id=847f6bef4e7b44f4b0adf1bda9dadb7e, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.272530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.272780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.274349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.284853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 803/1598] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:24.285710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.294764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd296c19-b832-496e-9927-fb172b19e5a1 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.300134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.300775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.301056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.301334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] PATH_INFO: `/v3/projects/b2eaefbf6c1842e590d71e1d94e32903/users/1559984d24f4441d915b84c791ee7548/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.316534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2eaefbf6c1842e590d71e1d94e32903, user_id=1559984d24f4441d915b84c791ee7548, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.319220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.345714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 796/1599] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:24 2022] PUT /identity/v3/projects/f0ca8c7ab1e64a0bb02a4941fe7b8105/users/847f6bef4e7b44f4b0adf1bda9dadb7e/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:24.346681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.352263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:24.409826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:24.411026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1b5e-28e2-486b-9e58-ceead79b57cb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.414779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 804/1600] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:24 2022] PUT /identity/v3/projects/b2eaefbf6c1842e590d71e1d94e32903/users/1559984d24f4441d915b84c791ee7548/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:24.417341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.476502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.477223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.477503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.477778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] PATH_INFO: `/v3/projects/4b092465a93d451d908e4727ab8fe634/users/dd810388eb0142318f9b9241fac0164a/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.485008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bb143-1c5e-4119-b401-1ffff242955b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.490299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.491106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.491385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.492009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.492563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.494168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.494483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b092465a93d451d908e4727ab8fe634, user_id=dd810388eb0142318f9b9241fac0164a, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.497252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.505373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 805/1601] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.505373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.515789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b0934-5415-43a3-b5bc-181d35a8b7ed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.521277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.521870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.522203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.522630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.523273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.524882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.538850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 806/1602] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:24.538850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.538850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:24.596122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0188d-d45e-4761-a980-1278025ac08b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.599718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed1bf2-5d52-4da0-86d4-2800cd99a15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.600333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed1bf2-5d52-4da0-86d4-2800cd99a15d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.600644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed1bf2-5d52-4da0-86d4-2800cd99a15d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.600939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed1bf2-5d52-4da0-86d4-2800cd99a15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.609812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed1bf2-5d52-4da0-86d4-2800cd99a15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.611243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed1bf2-5d52-4da0-86d4-2800cd99a15d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.615993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 797/1603] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:24 2022] PUT /identity/v3/projects/4b092465a93d451d908e4727ab8fe634/users/dd810388eb0142318f9b9241fac0164a/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:24.617174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.630746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 807/1604] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:24.632196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c181b19-6cb1-4455-8b09-3fa152708e0f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.632596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c181b19-6cb1-4455-8b09-3fa152708e0f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.632878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c181b19-6cb1-4455-8b09-3fa152708e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.655320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7c181b19-6cb1-4455-8b09-3fa152708e0f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:24.656530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7c181b19-6cb1-4455-8b09-3fa152708e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:24.689015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba470e-3729-4db7-86cf-6c1c7b57963c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.693745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.694431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.694738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.695017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] PATH_INFO: `/v3/projects/8ae9ee69453a4104b735cf758efeff61/users/f02bafa46fc6496796961af25a89910b/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.714586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae9ee69453a4104b735cf758efeff61, user_id=f02bafa46fc6496796961af25a89910b, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.717659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.723257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 808/1605] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:24.724440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.724746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.725003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.772164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] MFA Rules not processed for user `1559984d24f4441d915b84c791ee7548`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:24.773847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:24.793434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:24.812194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 798/1606] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:24 2022] PUT /identity/v3/projects/8ae9ee69453a4104b735cf758efeff61/users/f02bafa46fc6496796961af25a89910b/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:24.813014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.836792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:24.855679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 809/1607] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:24.857030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:24.873323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136a17a-ae82-417a-8e90-bf582aae615d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.895558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72104530-ac8a-48c6-b900-5c9fd7cf96aa admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:24.902127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caafd2-f4cc-4f1f-8e31-9bd28cf7ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.902879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caafd2-f4cc-4f1f-8e31-9bd28cf7ee3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.903157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caafd2-f4cc-4f1f-8e31-9bd28cf7ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.903430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caafd2-f4cc-4f1f-8e31-9bd28cf7ee3f admin admin] PATH_INFO: `/v3/projects/d093224fbda4411aa6ddae986e18b587/users/6c77a5189ed14e55b43a5f6eaf860db5/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.919524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caafd2-f4cc-4f1f-8e31-9bd28cf7ee3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d093224fbda4411aa6ddae986e18b587, user_id=6c77a5189ed14e55b43a5f6eaf860db5, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.922909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caafd2-f4cc-4f1f-8e31-9bd28cf7ee3f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.937907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:24.938836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:24.938957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:24.939334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] PATH_INFO: `/v3/projects/91b7336a644a46ff9a0c5b55a16169b9/users/d5f86863da7d4aa39e65f884774c820d/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:24.954198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b7336a644a46ff9a0c5b55a16169b9, user_id=d5f86863da7d4aa39e65f884774c820d, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:24.960787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:24.998227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 799/1608] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:24 2022] PUT /identity/v3/projects/d093224fbda4411aa6ddae986e18b587/users/6c77a5189ed14e55b43a5f6eaf860db5/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:24.999947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b121a3-cc03-48f2-9093-980aeb5d1a88 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.000300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b121a3-cc03-48f2-9093-980aeb5d1a88 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.001030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b121a3-cc03-48f2-9093-980aeb5d1a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.037958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c1b121a3-cc03-48f2-9093-980aeb5d1a88 None None] MFA Rules not processed for user `dd810388eb0142318f9b9241fac0164a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:25.040109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c1b121a3-cc03-48f2-9093-980aeb5d1a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:25.085156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 810/1609] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:24 2022] PUT /identity/v3/projects/91b7336a644a46ff9a0c5b55a16169b9/users/d5f86863da7d4aa39e65f884774c820d/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:25.086224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:25.138540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 800/1610] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:25.143968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e384e0b-dbc0-4d93-99cd-307323dabb90 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.144343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e384e0b-dbc0-4d93-99cd-307323dabb90 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.144767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e384e0b-dbc0-4d93-99cd-307323dabb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.163442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740915a1-5b52-4f74-979a-c784a4aa40db admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:25.167360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde9f94-6252-4e0b-a6df-9383582437ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:25.168079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde9f94-6252-4e0b-a6df-9383582437ee service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.168454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde9f94-6252-4e0b-a6df-9383582437ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.168788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde9f94-6252-4e0b-a6df-9383582437ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.180587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3e384e0b-dbc0-4d93-99cd-307323dabb90 None None] MFA Rules not processed for user `f02bafa46fc6496796961af25a89910b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:25.181802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3e384e0b-dbc0-4d93-99cd-307323dabb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:25.228877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:25.241322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde9f94-6252-4e0b-a6df-9383582437ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:25.242928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde9f94-6252-4e0b-a6df-9383582437ee service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:25.257438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 811/1611] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:25.259002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950073e1-5640-4c5c-9e22-59d3019ada5e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.259293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950073e1-5640-4c5c-9e22-59d3019ada5e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.263677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950073e1-5640-4c5c-9e22-59d3019ada5e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.263677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 812/1612] 10.222.0.46 () {54 vars in 962 bytes} [Wed Sep 7 19:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:25.263677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb89cf-777c-4f4c-ae52-97bb5344bec2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.263677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb89cf-777c-4f4c-ae52-97bb5344bec2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.263677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb89cf-777c-4f4c-ae52-97bb5344bec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.278752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 801/1613] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:25.278752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.278752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.278752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.315030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-60bb89cf-777c-4f4c-ae52-97bb5344bec2 None None] MFA Rules not processed for user `6c77a5189ed14e55b43a5f6eaf860db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:25.317764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60bb89cf-777c-4f4c-ae52-97bb5344bec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:25.322084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] MFA Rules not processed for user `d5f86863da7d4aa39e65f884774c820d`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:25.323662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:25.334497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:25.408914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 802/1614] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:25.410223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:25.420581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 813/1615] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:25.420581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e968a-c422-4076-b7d0-3f5deff88f91 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.420581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e968a-c422-4076-b7d0-3f5deff88f91 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.420818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e968a-c422-4076-b7d0-3f5deff88f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.440521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5573fe-fcba-4bbe-ab49-8079a9ba419d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:25.446857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:25.447634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.448010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.448541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.449290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:25.451442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:25.469643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b67e968a-c422-4076-b7d0-3f5deff88f91 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:25.470921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b67e968a-c422-4076-b7d0-3f5deff88f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:25.538915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 814/1616] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:25.547415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e730a7-ded8-4368-9d5c-50005f8857f8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.549722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e730a7-ded8-4368-9d5c-50005f8857f8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.549722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e730a7-ded8-4368-9d5c-50005f8857f8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.549722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 815/1617] 10.222.0.46 () {54 vars in 962 bytes} [Wed Sep 7 19:14:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:25.550122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e730a7-ded8-4368-9d5c-50005f8857f8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:25.551624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 803/1618] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:25.552659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:25.630945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d93cd-e0db-4ea2-9558-cb9935cc7171 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:25.638722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:25.638955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e730a7-ded8-4368-9d5c-50005f8857f8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:25.639462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.639797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.640012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.640883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:25.643340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:25.647222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:25.648346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.649323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.659278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.666868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:25.723656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:25.725104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:25.742029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 816/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:25.743651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:25.747508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 804/1620] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:25.748430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:25.826250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3c479-0a6a-4466-80a1-b283f4a3ccf8 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:25.843376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13475f0d-485b-499d-8088-90acc4c76841 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:25.847805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefba7f5-5708-496a-99e8-7f285df7a988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:25.848779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefba7f5-5708-496a-99e8-7f285df7a988 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.849448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefba7f5-5708-496a-99e8-7f285df7a988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.850248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefba7f5-5708-496a-99e8-7f285df7a988 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.851260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefba7f5-5708-496a-99e8-7f285df7a988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:25.853736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefba7f5-5708-496a-99e8-7f285df7a988 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:25.947195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:25.948645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.948931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.949208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.963858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 805/1621] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:25.963858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:25.963858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:25.963858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:25.969591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:25.985549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:25.986806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:26.016491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 806/1622] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Sep 7 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:26.017389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:26.017928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.033672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.037076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.040721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be8694-b6e6-4d7e-b94a-fbfdcf3389e7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.054152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 817/1623] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:26.055668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.070156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660baa7c-8065-4cfc-b281-5986fcb7a562 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.075580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.076363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.076653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.076930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.077739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.081090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.082275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.083871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.084222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.084566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.085356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.088536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.108782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 818/1624] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:26.108997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.142171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615038f-ac79-4697-858d-f4f82f09abac admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.157705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.158361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.162462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.162462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.226910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 807/1625] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:26 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:26.234620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.262493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.264936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.267009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de86efb-e548-4af5-9bdb-0601b246abe2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.278602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 819/1626] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:26.279814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.285454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa98ea-35f4-4a5c-9f76-f05f2c433fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.287065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa98ea-35f4-4a5c-9f76-f05f2c433fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.288289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa98ea-35f4-4a5c-9f76-f05f2c433fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.289470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa98ea-35f4-4a5c-9f76-f05f2c433fc4 admin admin] PATH_INFO: `/v3/projects/492178b10d464227922ba76916f58219/users/e21f68fbf09b49f5b8f27d0d00bce634/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.294390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7efe0f-290d-4d65-aacd-e0a872c0c9c1 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.301198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.301905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.302245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.302802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.303820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.305656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.321170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 820/1627] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:26.322189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.334359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5a571-a82f-440f-a397-a4ae4f07f029 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.335431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa98ea-35f4-4a5c-9f76-f05f2c433fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=492178b10d464227922ba76916f58219, user_id=e21f68fbf09b49f5b8f27d0d00bce634, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.343311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa98ea-35f4-4a5c-9f76-f05f2c433fc4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.346479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caab04-28a6-46a9-b6f8-622cf8b14666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.347262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caab04-28a6-46a9-b6f8-622cf8b14666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.347541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caab04-28a6-46a9-b6f8-622cf8b14666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.347811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caab04-28a6-46a9-b6f8-622cf8b14666 admin admin] PATH_INFO: `/v3/projects/66bdde05739743f29d7abb4219fbebdb/users/f686591cef6f4f389175758127bae39f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.400902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caab04-28a6-46a9-b6f8-622cf8b14666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bdde05739743f29d7abb4219fbebdb, user_id=f686591cef6f4f389175758127bae39f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.407047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caab04-28a6-46a9-b6f8-622cf8b14666 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.425269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 808/1628] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:26 2022] PUT /identity/v3/projects/492178b10d464227922ba76916f58219/users/e21f68fbf09b49f5b8f27d0d00bce634/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:26.426700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267cf2-a8ac-4754-8d65-340c54f80957 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.426991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267cf2-a8ac-4754-8d65-340c54f80957 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.427265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267cf2-a8ac-4754-8d65-340c54f80957 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.430946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 809/1629] 10.222.0.46 () {54 vars in 912 bytes} [Wed Sep 7 19:14:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:26.431744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267cf2-a8ac-4754-8d65-340c54f80957 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.523415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267cf2-a8ac-4754-8d65-340c54f80957 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.529857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.531118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.531499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.531786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.532597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.536484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.554536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 821/1630] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:26 2022] PUT /identity/v3/projects/66bdde05739743f29d7abb4219fbebdb/users/f686591cef6f4f389175758127bae39f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:26.555820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 810/1631] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:26.556822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.559284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d5cc1-a6d0-48e5-b2ad-5b38bfa098e3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.559568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d5cc1-a6d0-48e5-b2ad-5b38bfa098e3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.559843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d5cc1-a6d0-48e5-b2ad-5b38bfa098e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.573397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2191fdb-f2b9-400a-b9bb-f890adbe0939 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.628206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9e7d5cc1-a6d0-48e5-b2ad-5b38bfa098e3 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:26.630203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9e7d5cc1-a6d0-48e5-b2ad-5b38bfa098e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:26.656601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.657647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.657973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.658916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] PATH_INFO: `/v3/projects/492178b10d464227922ba76916f58219/users/e21f68fbf09b49f5b8f27d0d00bce634/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.662489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=492178b10d464227922ba76916f58219, user_id=e21f68fbf09b49f5b8f27d0d00bce634, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.666685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.694124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 811/1632] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:26 2022] PUT /identity/v3/projects/492178b10d464227922ba76916f58219/users/e21f68fbf09b49f5b8f27d0d00bce634/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:26.695814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.793693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 822/1633] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:26.795298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749cd4-3a21-4f48-b26d-989616933a8a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.795638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749cd4-3a21-4f48-b26d-989616933a8a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.795920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749cd4-3a21-4f48-b26d-989616933a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.807468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:26.837968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d3749cd4-3a21-4f48-b26d-989616933a8a None None] MFA Rules not processed for user `e21f68fbf09b49f5b8f27d0d00bce634`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:26.839376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d3749cd4-3a21-4f48-b26d-989616933a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:26.847278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d32e-f5b6-4e36-a079-7aaf1fe72b4c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.858841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.860017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.860297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.860730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.861910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.863783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:26.866135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:26.943277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 823/1634] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:26.945238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af335cd-1015-4f69-b7cc-10630869f0be None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.946175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af335cd-1015-4f69-b7cc-10630869f0be None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.946625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af335cd-1015-4f69-b7cc-10630869f0be None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.947907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 812/1635] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:26.949756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.955982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 824/1636] 10.222.0.46 () {54 vars in 912 bytes} [Wed Sep 7 19:14:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:26.957911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af335cd-1015-4f69-b7cc-10630869f0be None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:26.973715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af335cd-1015-4f69-b7cc-10630869f0be None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:26.980620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c07a1c-49f0-42fb-bdb6-1c34d17f0564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:26.981493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07a1c-49f0-42fb-bdb6-1c34d17f0564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:26.981971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07a1c-49f0-42fb-bdb6-1c34d17f0564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:26.982524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07a1c-49f0-42fb-bdb6-1c34d17f0564 admin admin] PATH_INFO: `/v3/projects/66bdde05739743f29d7abb4219fbebdb/users/f686591cef6f4f389175758127bae39f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:26.986019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c07a1c-49f0-42fb-bdb6-1c34d17f0564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bdde05739743f29d7abb4219fbebdb, user_id=f686591cef6f4f389175758127bae39f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:26.989466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c07a1c-49f0-42fb-bdb6-1c34d17f0564 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:27.025550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 825/1637] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:26 2022] PUT /identity/v3/projects/66bdde05739743f29d7abb4219fbebdb/users/f686591cef6f4f389175758127bae39f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:27.025550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afe78-a10e-4b8e-9738-b4fbfb3ae4b9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:27.025550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afe78-a10e-4b8e-9738-b4fbfb3ae4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:27.025550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afe78-a10e-4b8e-9738-b4fbfb3ae4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:27.064204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b89afe78-a10e-4b8e-9738-b4fbfb3ae4b9 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:27.070143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b89afe78-a10e-4b8e-9738-b4fbfb3ae4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:27.092487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a9247-8524-4ff1-bfd8-7bb7caf341ae admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:27.178517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:27.179353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:27.179730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:27.180170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:27.237466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 826/1638] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:14:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:27.247135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:27.247629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:27.248034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:27.291126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] MFA Rules not processed for user `f686591cef6f4f389175758127bae39f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:27.293470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:27.328065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:27.328065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:27.371313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 813/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:27.372291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:27.374897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 827/1640] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:27.397621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccac16-89e7-48ac-bbc2-8e36ff89638e service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:27.405698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:27.406997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:27.407284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:27.407557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:27.552743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:27.554216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:27.576308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 814/1641] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:27.728619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:27.836762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:27.852232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19e4ed-61ef-45d7-838d-9dcb6bbbaffb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:27.857051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:27.860583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:27.860583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:27.860583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:27.872312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:27.873845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:27.886895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 828/1642] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:28.738911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:28.753507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710bca-fbe7-41d6-855b-37655ff28a14 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:28.757945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:28.758758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:28.761713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:28.767492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:28.799220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:28.800662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:28.838684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 815/1643] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:29.408472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:29.418934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762f619-25fe-44cf-aa8b-14a8c2d0469b service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:29.425148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df9b4-8a51-491c-aa71-4a3dacb6575e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:29.426195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df9b4-8a51-491c-aa71-4a3dacb6575e service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:29.426690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df9b4-8a51-491c-aa71-4a3dacb6575e service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:29.427108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df9b4-8a51-491c-aa71-4a3dacb6575e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:29.456862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:29.619681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73df9b4-8a51-491c-aa71-4a3dacb6575e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:29.625961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73df9b4-8a51-491c-aa71-4a3dacb6575e service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:29.666273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 829/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:31.886255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:32.039400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9d9a3-ae75-4b35-bd9f-c3fe435cac5f service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:32.050715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:32.051472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:32.051824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:32.052282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] PATH_INFO: `/v3/users/dd810388eb0142318f9b9241fac0164a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:32.053520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd810388eb0142318f9b9241fac0164a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:32.055285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:14:32.055285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Sep 07 19:14:32.055285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: Sep 07 19:14:32.056118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:32.488044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd810388eb0142318f9b9241fac0164a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:32.488886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd810388eb0142318f9b9241fac0164a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:32.498530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd810388eb0142318f9b9241fac0164a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:32.528498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd810388eb0142318f9b9241fac0164a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:32.580054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd810388eb0142318f9b9241fac0164a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:32.684222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 816/1645] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:31 2022] DELETE /identity/v3/users/dd810388eb0142318f9b9241fac0164a => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:32.689171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:32.689693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:32.689945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:32.730890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:32.732229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:32.858061 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 830/1646] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:32.913907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:32.916094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:33.012582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:33.050068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b4fa5-a084-4a20-b3d9-ececf21b89af admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:33.065096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9350c33-8b73-47f1-afe0-088cc4800daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:33.065921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9350c33-8b73-47f1-afe0-088cc4800daa service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:33.066304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9350c33-8b73-47f1-afe0-088cc4800daa service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:33.066716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9350c33-8b73-47f1-afe0-088cc4800daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:33.086518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9350c33-8b73-47f1-afe0-088cc4800daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:33.088126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9350c33-8b73-47f1-afe0-088cc4800daa service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:33.113321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 817/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:33.254608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:33.254608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:33.280022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c477eba-f204-42fc-849e-3c72d5d1ae71 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:33.300595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:33.300595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:33.300595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:33.300595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] PATH_INFO: `/v3/users/f02bafa46fc6496796961af25a89910b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:33.303831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02bafa46fc6496796961af25a89910b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:33.307267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:33.625737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02bafa46fc6496796961af25a89910b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:33.631370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02bafa46fc6496796961af25a89910b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:33.631937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02bafa46fc6496796961af25a89910b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:33.645640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02bafa46fc6496796961af25a89910b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:33.674560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02bafa46fc6496796961af25a89910b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:33.770384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 831/1648] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:33 2022] DELETE /identity/v3/users/f02bafa46fc6496796961af25a89910b => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:33.774966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:33.775710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:33.776189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:33.821281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:33.824374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:34.031646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 818/1649] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:34.041399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:34.043305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:34.097188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:34.198041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed125f9-45ef-4523-9339-dbdea2bfb7d0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:34.213522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:34.214169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:34.214455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:34.214808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:34.238854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:34.240365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:34.279679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 832/1650] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:35.030514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:35.033218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:35.061699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60690eaa-56ea-45a3-a335-ff66349f936a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:35.085395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:35.086081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:35.086366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:35.086676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] PATH_INFO: `/v3/projects/4b092465a93d451d908e4727ab8fe634` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:35.088277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b092465a93d451d908e4727ab8fe634)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:35.091821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:35.280574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b092465a93d451d908e4727ab8fe634', 'request_id': 'req-732251c4-3398-4c0d-b876-3e9f8b51cf25'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:35.316380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b092465a93d451d908e4727ab8fe634', 'request_id': 'req-732251c4-3398-4c0d-b876-3e9f8b51cf25'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:35.344326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b092465a93d451d908e4727ab8fe634', 'request_id': 'req-732251c4-3398-4c0d-b876-3e9f8b51cf25'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:35.452771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] The token cache is being invalidate because project 4b092465a93d451d908e4727ab8fe634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:35.453359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:35.454197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-732251c4-3398-4c0d-b876-3e9f8b51cf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:35.457729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 819/1651] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:35 2022] DELETE /identity/v3/projects/4b092465a93d451d908e4727ab8fe634 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:35.462701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:35.489520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd235d9-ee21-433d-ab00-985a092ec48f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:35.634121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:35.634758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:35.635235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:35.709400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] MFA Rules not processed for user `847f6bef4e7b44f4b0adf1bda9dadb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:35.710816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:35.715510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:35.716262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:35.717196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:35.717463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] PATH_INFO: `/v3/users/e21f68fbf09b49f5b8f27d0d00bce634` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:35.719725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21f68fbf09b49f5b8f27d0d00bce634)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:35.721804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:35.836221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:35.856242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 820/1652] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:35.946567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:35.946567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:35.946567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674b6d0-e64b-4b42-8e65-a2a87c5893a1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:35.948317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:35.948317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:35.948317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:35.948317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] PATH_INFO: `/v3/projects/8ae9ee69453a4104b735cf758efeff61` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:35.948853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae9ee69453a4104b735cf758efeff61)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:35.948853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:36.202444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae9ee69453a4104b735cf758efeff61', 'request_id': 'req-69939176-bb86-4f56-8892-064eade0472d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.213465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21f68fbf09b49f5b8f27d0d00bce634'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.216067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae9ee69453a4104b735cf758efeff61', 'request_id': 'req-69939176-bb86-4f56-8892-064eade0472d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.216647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21f68fbf09b49f5b8f27d0d00bce634'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.219488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21f68fbf09b49f5b8f27d0d00bce634'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.234778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae9ee69453a4104b735cf758efeff61', 'request_id': 'req-69939176-bb86-4f56-8892-064eade0472d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.257228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21f68fbf09b49f5b8f27d0d00bce634'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.279538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8147601b-7f96-49b9-a120-d04429e342f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21f68fbf09b49f5b8f27d0d00bce634'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.331999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] The token cache is being invalidate because project 8ae9ee69453a4104b735cf758efeff61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:36.332473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.333322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-69939176-bb86-4f56-8892-064eade0472d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:36.338113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 821/1653] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:35 2022] DELETE /identity/v3/projects/8ae9ee69453a4104b735cf758efeff61 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:36.339857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:36.340433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:36.340923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:36.365676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] MFA Rules not processed for user `847f6bef4e7b44f4b0adf1bda9dadb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:36.367436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:36.422787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 833/1654] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:35 2022] DELETE /identity/v3/users/e21f68fbf09b49f5b8f27d0d00bce634 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:36.424786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6a74e-30fe-4eb6-bf28-5961e2af4dd6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:36.425346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6a74e-30fe-4eb6-bf28-5961e2af4dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:36.428320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6a74e-30fe-4eb6-bf28-5961e2af4dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:36.508405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d9a6a74e-30fe-4eb6-bf28-5961e2af4dd6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:36.522385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9a6a74e-30fe-4eb6-bf28-5961e2af4dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:36.545342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 822/1655] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:36.560950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:36.563345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:36.752488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 834/1656] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:36.758804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:36.759443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:36.760031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:36.822468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:36.824003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:36.842819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 835/1657] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:36.844938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:36.866437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8257c-9419-4f29-b49f-1cd9a3688d66 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:36.880027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:36.880913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:36.881653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:36.882360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:36.885186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:36.885991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:36.891139 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:36.894084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:36.897128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7ad48-e3bd-4e4c-9af0-e4429f9851c6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:36.949708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 836/1658] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:36.960126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:37.030039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b892bf-2d43-4554-ab21-4e6061328221 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:37.063536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:37.071464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.072236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.077603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.079564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:37.083050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:37.181573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:37.182212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.182491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.182999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.185036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:37.273031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 837/1659] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:36 2022] POST /identity/v3/projects => generated 373 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:37.276961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:37.361507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:37.363424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:37.382832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 823/1660] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 822 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:14:37.509113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba87428-b3ca-475c-9a7f-fafd2b3b0750 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:37.521510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f031e-12b5-4ba8-b642-5699ec52fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:37.523067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f031e-12b5-4ba8-b642-5699ec52fb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.524396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f031e-12b5-4ba8-b642-5699ec52fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.525599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f031e-12b5-4ba8-b642-5699ec52fb7f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.528098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f031e-12b5-4ba8-b642-5699ec52fb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:37.534838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f031e-12b5-4ba8-b642-5699ec52fb7f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:37.638524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 838/1661] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:37 2022] POST /identity/v3/users => generated 287 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:37.644142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:37.656492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.658322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.658779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.688832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f503b-f5b2-40ba-8d48-4664e1fe9ce4 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:37.704974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8f6c3-80d7-4b39-acd5-925095881226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:37.712681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8f6c3-80d7-4b39-acd5-925095881226 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.712681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8f6c3-80d7-4b39-acd5-925095881226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.712681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8f6c3-80d7-4b39-acd5-925095881226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.712681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8f6c3-80d7-4b39-acd5-925095881226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:37.712681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8f6c3-80d7-4b39-acd5-925095881226 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:37.742134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 824/1662] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:37.743688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.744139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.744660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.765906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] MFA Rules not processed for user `920bdd998f0c4b62b108afc799a43964`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:37.767193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:37.800895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] MFA Rules not processed for user `ddf07a7d8aed4b3faf058c66347cf315`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:37.802144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:37.950362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 839/1663] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:37.951495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:37.966274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63908d8-5ff0-45f4-9c3e-fb0d45c52217 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:37.974408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34293f-2dca-4714-9f9c-c2ae00f0ff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:37.976168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34293f-2dca-4714-9f9c-c2ae00f0ff80 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:37.976168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34293f-2dca-4714-9f9c-c2ae00f0ff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:37.976168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34293f-2dca-4714-9f9c-c2ae00f0ff80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:37.976704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34293f-2dca-4714-9f9c-c2ae00f0ff80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:37.978757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34293f-2dca-4714-9f9c-c2ae00f0ff80 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:37.979091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 825/1664] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:37.980111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:37.989908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.009318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:38.018726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ec3c9-c1d1-4cfb-8541-dac275db991c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.023867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:38.025214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.025214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.026121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] PATH_INFO: `/v3/projects/492178b10d464227922ba76916f58219` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.028047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=492178b10d464227922ba76916f58219)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:38.029486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 840/1665] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:38.031146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.031595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.032014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.033979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:38.075419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] MFA Rules not processed for user `ddf07a7d8aed4b3faf058c66347cf315`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:38.086023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.115211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 841/1666] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:38.117571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:38.154673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba86d4-b7d2-4f99-b45b-49081f4afee6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.170727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63325626-e70b-48c6-a731-a648b9adaa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:38.170727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325626-e70b-48c6-a731-a648b9adaa70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.170727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325626-e70b-48c6-a731-a648b9adaa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.170727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325626-e70b-48c6-a731-a648b9adaa70 admin admin] PATH_INFO: `/v3/projects/332a6aeb4ada4c7985045bcf95f38ed5/users/f031c535798743a6ad2114d8c0e05d76/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.185350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63325626-e70b-48c6-a731-a648b9adaa70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332a6aeb4ada4c7985045bcf95f38ed5, user_id=f031c535798743a6ad2114d8c0e05d76, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:38.190998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63325626-e70b-48c6-a731-a648b9adaa70 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:38.209388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492178b10d464227922ba76916f58219', 'request_id': 'req-6a523e35-f60c-4d36-b588-6a15dd1c58ec'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:38.228974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492178b10d464227922ba76916f58219', 'request_id': 'req-6a523e35-f60c-4d36-b588-6a15dd1c58ec'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:38.253469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492178b10d464227922ba76916f58219', 'request_id': 'req-6a523e35-f60c-4d36-b588-6a15dd1c58ec'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:38.300843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 842/1667] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:38 2022] PUT /identity/v3/projects/332a6aeb4ada4c7985045bcf95f38ed5/users/f031c535798743a6ad2114d8c0e05d76/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:38.302643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd88d3-e50a-4f29-a8ef-c9b6fcbe7f06 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.303134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd88d3-e50a-4f29-a8ef-c9b6fcbe7f06 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.303643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd88d3-e50a-4f29-a8ef-c9b6fcbe7f06 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.305865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 843/1668] 10.222.0.46 () {54 vars in 912 bytes} [Wed Sep 7 19:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:38.312596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c55d46-cd9b-41f5-b94f-446ba50a9acb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.313097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c55d46-cd9b-41f5-b94f-446ba50a9acb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.313518 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c55d46-cd9b-41f5-b94f-446ba50a9acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.358005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d6c55d46-cd9b-41f5-b94f-446ba50a9acb None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:38.364204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6c55d46-cd9b-41f5-b94f-446ba50a9acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.368080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] The token cache is being invalidate because project 492178b10d464227922ba76916f58219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:38.375114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:38.375114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6a523e35-f60c-4d36-b588-6a15dd1c58ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:38.375114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 826/1669] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:37 2022] DELETE /identity/v3/projects/492178b10d464227922ba76916f58219 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:38.378467 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.378991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.379830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.438964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] MFA Rules not processed for user `f031c535798743a6ad2114d8c0e05d76`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:38.440807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.469666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 844/1670] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:14:38.471125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.471434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.471714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.508159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:38.509670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.529743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 827/1671] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:38.530465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:38.535530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.576355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:38.597871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 845/1672] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:38.599732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:38.601549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.617261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a0010-e3c4-47a1-8abb-7918cf73ae99 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.621497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93901e74-28b5-4075-a532-3dbf95fefff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:38.622264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901e74-28b5-4075-a532-3dbf95fefff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.622583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901e74-28b5-4075-a532-3dbf95fefff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.623370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901e74-28b5-4075-a532-3dbf95fefff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.627016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642f3d-2598-4ad3-916e-625d0a4468c0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.631926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93901e74-28b5-4075-a532-3dbf95fefff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:38.632834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:38.633435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93901e74-28b5-4075-a532-3dbf95fefff5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:38.633543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.633874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.634212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.636080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.650772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 846/1673] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:38.652697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.653182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.654253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.681318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:38.686158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:38.697746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 847/1674] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:38.703081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:38.710305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:38.711965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:38.715565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e90954-8333-4e66-99cf-02efae9006b6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.722502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:38.723477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.723900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.724321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.724620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 828/1675] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:38.725660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:38.726129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:38.728205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:38.752365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 848/1676] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:38.752567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:38.769597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271cc02-39d6-4b43-89c9-47db2e42ebce service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.775868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:38.776588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:38.776937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:38.777350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:38.779067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:38.782348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:38.895168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 829/1677] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:38.896591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:38.908636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41858657-52f3-40d6-9e4d-53d9cf990b8b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:38.947286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.012027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.013434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.013738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.014013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.015598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.020219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.065033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91654e7-9677-430d-9712-fefe95b04056 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:39.076111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.091986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.091986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.091986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.091986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:39.162245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.164022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.174456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 830/1678] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:14:39.206544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 849/1679] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:38 2022] POST /identity/v3/users => generated 294 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:39.207950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.224983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29daddfe-0277-4aec-9115-47bb51235e7e service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:39.241263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.241263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.241263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.241263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.241586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.243072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.265806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 831/1680] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:39.270303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.283169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1110aa-35a7-43d9-b860-585fc305b1c1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:39.290013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248900d-bfb3-4219-a71b-3fbfe6a8f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.290917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248900d-bfb3-4219-a71b-3fbfe6a8f346 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.291668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248900d-bfb3-4219-a71b-3fbfe6a8f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.292612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248900d-bfb3-4219-a71b-3fbfe6a8f346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.293547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248900d-bfb3-4219-a71b-3fbfe6a8f346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.297378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248900d-bfb3-4219-a71b-3fbfe6a8f346 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.312848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 850/1681] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:39.317227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.328467 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31333c7-d689-42a0-9fd2-1d6c4818da18 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:39.333557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.334361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.335248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.335678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] PATH_INFO: `/v3/projects/27bf48c0ef964de3ba19a10b74fa6ee1/users/f4aa6288f84349c7883a046f56b77472/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.362971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bf48c0ef964de3ba19a10b74fa6ee1, user_id=f4aa6288f84349c7883a046f56b77472, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.366524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.371630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:39.458181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 832/1682] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:39 2022] PUT /identity/v3/projects/27bf48c0ef964de3ba19a10b74fa6ee1/users/f4aa6288f84349c7883a046f56b77472/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:39.468950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe5f5-b47b-4db3-8600-dc8cee190b8f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.469277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe5f5-b47b-4db3-8600-dc8cee190b8f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.470317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe5f5-b47b-4db3-8600-dc8cee190b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.482300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:39.519137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-edfbe5f5-b47b-4db3-8600-dc8cee190b8f None None] MFA Rules not processed for user `f4aa6288f84349c7883a046f56b77472`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:39.527693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-edfbe5f5-b47b-4db3-8600-dc8cee190b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:39.553044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.555220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:39.681564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 851/1683] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:39.683026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d453a5-4ea3-4b4f-ac29-f32322cca032 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.683453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d453a5-4ea3-4b4f-ac29-f32322cca032 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.683875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d453a5-4ea3-4b4f-ac29-f32322cca032 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.685414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154dc88-462d-48db-a8e5-535a920ca11f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:39.692488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 852/1684] 10.222.0.46 () {54 vars in 912 bytes} [Wed Sep 7 19:14:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:39.693310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d453a5-4ea3-4b4f-ac29-f32322cca032 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.693515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.694751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.695448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.695924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.698152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:39.715909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.718868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19cb59-c8f8-4849-ba1a-b8d8b6eb38d4 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.737940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 833/1685] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:14:39.739472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.739918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.740207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.771226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d453a5-4ea3-4b4f-ac29-f32322cca032 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:39.777726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:39.779447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:39.781260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:39.781949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:39.782875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:39.783158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:39.783815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:39.791127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:39.880173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 834/1686] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:14:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:39.881118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:39.885559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:39.937300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 853/1687] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:39 2022] POST /identity/v3/projects => generated 387 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:39.943056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:40.045661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103a9db-53c9-45fd-ba5d-c7b3117e9b00 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:40.052458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35672f68-30df-4e6c-a687-0a3c78fc4dab None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:40.063431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:40.064462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.064978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.065579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.100354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:40.103460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:40.115153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:40.163490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 854/1688] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:40.164498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:40.219031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:40.219717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.219995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.220256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.312646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:40.314418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:40.326070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 835/1689] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:40.343857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff415-5772-41b7-8b58-4e2144e0a18b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:40.372134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:40.372134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.372134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.372134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.375438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:40.377777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:40.392440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:40.482358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 855/1690] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:40 2022] POST /identity/v3/users => generated 300 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:40.482880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:40.500697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ccfbb-2a01-47c3-b56a-639212db807c service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:40.525369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915f0b4-ab96-48a4-91bb-813633b5a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:40.528743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915f0b4-ab96-48a4-91bb-813633b5a7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.530588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915f0b4-ab96-48a4-91bb-813633b5a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.530588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915f0b4-ab96-48a4-91bb-813633b5a7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.532406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915f0b4-ab96-48a4-91bb-813633b5a7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:40.537223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915f0b4-ab96-48a4-91bb-813633b5a7a5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:40.556993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 836/1691] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:40.560649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:40.576777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.579241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.579597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.581129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763405a-ac89-491e-a6c2-36f593161cce admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:40.595638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:40.596406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.596686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.596960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] PATH_INFO: `/v3/projects/4c8e1d524f654d1a8e52fd54b029e3ca/users/f7ec5b177414433a992a163a9dbc5979/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.630597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8e1d524f654d1a8e52fd54b029e3ca, user_id=f7ec5b177414433a992a163a9dbc5979, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:40.633537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:40.646970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] MFA Rules not processed for user `847f6bef4e7b44f4b0adf1bda9dadb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:40.648221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:40.729174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 856/1692] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:40 2022] PUT /identity/v3/projects/4c8e1d524f654d1a8e52fd54b029e3ca/users/f7ec5b177414433a992a163a9dbc5979/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:40.732874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:40.839587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 837/1693] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:40.980052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111dc13-11db-4081-8add-f3f458fde0d7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:40.993348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4cb7-cd64-45aa-8ad9-2952f72c98ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:40.997551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4cb7-cd64-45aa-8ad9-2952f72c98ed admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:40.997551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4cb7-cd64-45aa-8ad9-2952f72c98ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:40.997551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4cb7-cd64-45aa-8ad9-2952f72c98ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:40.997551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c4cb7-cd64-45aa-8ad9-2952f72c98ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:41.002917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c4cb7-cd64-45aa-8ad9-2952f72c98ed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:41.018665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 857/1694] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:14:41.022695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:41.044310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50283eda-dd5b-4522-9dca-163aab459d29 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:41.055940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:41.056619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:41.056905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:41.057180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] PATH_INFO: `/v3/projects/4c8e1d524f654d1a8e52fd54b029e3ca/users/f7ec5b177414433a992a163a9dbc5979/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:41.067606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8e1d524f654d1a8e52fd54b029e3ca, user_id=f7ec5b177414433a992a163a9dbc5979, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:41.070878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:41.090299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 838/1695] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:41 2022] PUT /identity/v3/projects/4c8e1d524f654d1a8e52fd54b029e3ca/users/f7ec5b177414433a992a163a9dbc5979/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:41.101629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:41.105394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:41.105394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:41.169413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] MFA Rules not processed for user `f7ec5b177414433a992a163a9dbc5979`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:41.172359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:41.306949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:41.385090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 858/1696] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:41.397197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:41.411634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:41.533926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ad403-735f-4d7d-a7d8-dedf956f6e2c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:41.550866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:41.551713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:41.551994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:41.552270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:41.600194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:41.601914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:41.625914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 839/1697] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:41.685044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:41.766409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:41.805572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:41.812333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:41.981667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de723-949a-4dfc-8272-84ddf4bc2215 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:42.000263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:42.001505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:42.001804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:42.013986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ee279-3fc2-402a-a0f6-f5e53ebf3ad0 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:42.033475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] PATH_INFO: `/v3/users/f4aa6288f84349c7883a046f56b77472` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:42.045449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4aa6288f84349c7883a046f56b77472)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:42.050206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:42.053992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:42.055031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:42.055644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] PATH_INFO: `/v3/users/f031c535798743a6ad2114d8c0e05d76` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:42.060270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f031c535798743a6ad2114d8c0e05d76)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:42.060390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:42.071229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:42.323597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4aa6288f84349c7883a046f56b77472'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.325760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4aa6288f84349c7883a046f56b77472'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.331070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4aa6288f84349c7883a046f56b77472'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.357071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4aa6288f84349c7883a046f56b77472'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.408188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4aa6288f84349c7883a046f56b77472'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.434481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f031c535798743a6ad2114d8c0e05d76'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.435683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f031c535798743a6ad2114d8c0e05d76'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.436741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f031c535798743a6ad2114d8c0e05d76'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.472404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f031c535798743a6ad2114d8c0e05d76'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.518853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 859/1698] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:41 2022] DELETE /identity/v3/users/f4aa6288f84349c7883a046f56b77472 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:42.520032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:42.524332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:42.539190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-083e1c08-b256-456a-b853-969fa0275c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f031c535798743a6ad2114d8c0e05d76'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:42.629663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 840/1699] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:41 2022] DELETE /identity/v3/users/f031c535798743a6ad2114d8c0e05d76 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:42.631323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be3c1-7e08-4465-9cb8-717fedd4ed04 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:42.631623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be3c1-7e08-4465-9cb8-717fedd4ed04 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:42.631892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be3c1-7e08-4465-9cb8-717fedd4ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:42.731161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d8810-2c9f-4dd3-9a52-d9f8740b40b5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:42.761873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-099be3c1-7e08-4465-9cb8-717fedd4ed04 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:42.765493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-099be3c1-7e08-4465-9cb8-717fedd4ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:42.984651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:43.003983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:43.004812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:43.005152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:43.005748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] PATH_INFO: `/v3/users/1559984d24f4441d915b84c791ee7548` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:43.013267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1559984d24f4441d915b84c791ee7548)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:43.014452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:43.015305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 841/1700] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:43.018163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:43.021636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:43.023088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:43.081534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:43.083313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.128384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:43.133982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 842/1701] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:43.135592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:43.138900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.323462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1559984d24f4441d915b84c791ee7548'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:43.326436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1559984d24f4441d915b84c791ee7548'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:43.328857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1559984d24f4441d915b84c791ee7548'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:43.352942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1559984d24f4441d915b84c791ee7548'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:43.354056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dc9c7-ec02-4dab-aed8-842ed858ffd3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:43.367360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:43.371341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:43.371634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:43.373272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:43.379896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.397592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4adc8863-f8f0-4daf-b3b0-1f5d67a8fd25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1559984d24f4441d915b84c791ee7548'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:43.415748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:43.422321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:43.445126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 843/1702] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:43.446358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:43.448439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.461837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cb192-ea32-4e11-8e45-c525cf9ace08 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:43.468377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:43.469360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:43.470113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:43.470450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:43.476235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.495013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 860/1703] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:42 2022] DELETE /identity/v3/users/1559984d24f4441d915b84c791ee7548 => generated 0 bytes in 972 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:43.497350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f5b4-3c52-48fa-bf9b-baaaa0cf8c36 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:43.497350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f5b4-3c52-48fa-bf9b-baaaa0cf8c36 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:43.497350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f5b4-3c52-48fa-bf9b-baaaa0cf8c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:43.529449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:43.534262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:43.572879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8874f5b4-3c52-48fa-bf9b-baaaa0cf8c36 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:43.576856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8874f5b4-3c52-48fa-bf9b-baaaa0cf8c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.580739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 844/1704] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:43.633514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:43.721918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 861/1705] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:43.747270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:43.753007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:43.769155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1175a4-0b56-4856-9dd7-90571427ad5c service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:43.918966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:43.921578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:43.921996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:43.922298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:43.935233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:43.937586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:43.968984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 845/1706] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:44.052693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:44.053544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:44.053544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:44.108027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] MFA Rules not processed for user `920bdd998f0c4b62b108afc799a43964`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:44.109878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:44.251326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 862/1707] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:44.615837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:44.862512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:45.255556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:45.258130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:45.289195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f5f6c-8664-4d4d-8c75-1b7a26858cdb service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:45.307485 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:45.311128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:45.311128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:45.311128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] PATH_INFO: `/v3/projects/b2eaefbf6c1842e590d71e1d94e32903` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:45.312132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2eaefbf6c1842e590d71e1d94e32903)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:45.314250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:45.394512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:45.397507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:45.433585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0f8fb-635b-4282-928e-c377632d88b6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:45.464883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:45.465817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:45.466337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:45.467153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] PATH_INFO: `/v3/projects/27bf48c0ef964de3ba19a10b74fa6ee1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:45.472908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bf48c0ef964de3ba19a10b74fa6ee1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:45.482535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 19:14:45.482535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 07 19:14:45.482535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: Sep 07 19:14:45.482535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:45.536943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2eaefbf6c1842e590d71e1d94e32903', 'request_id': 'req-cb320177-68cb-4507-980b-5013804f5cdb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.552359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2eaefbf6c1842e590d71e1d94e32903', 'request_id': 'req-cb320177-68cb-4507-980b-5013804f5cdb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.567496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2eaefbf6c1842e590d71e1d94e32903', 'request_id': 'req-cb320177-68cb-4507-980b-5013804f5cdb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.711134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] The token cache is being invalidate because project b2eaefbf6c1842e590d71e1d94e32903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:45.711875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.718236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.720214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 846/1708] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:45 2022] DELETE /identity/v3/projects/b2eaefbf6c1842e590d71e1d94e32903 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:45.721295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:45.724415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:45.780121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bf48c0ef964de3ba19a10b74fa6ee1', 'request_id': 'req-ae994dad-4355-4d74-8a2b-1c03fa6090e4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.871068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bf48c0ef964de3ba19a10b74fa6ee1', 'request_id': 'req-ae994dad-4355-4d74-8a2b-1c03fa6090e4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.902616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bf48c0ef964de3ba19a10b74fa6ee1', 'request_id': 'req-ae994dad-4355-4d74-8a2b-1c03fa6090e4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.915974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] The token cache is being invalidate because project 27bf48c0ef964de3ba19a10b74fa6ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:45.916551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.917569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae994dad-4355-4d74-8a2b-1c03fa6090e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:45.926928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 863/1709] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:45 2022] DELETE /identity/v3/projects/27bf48c0ef964de3ba19a10b74fa6ee1 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:45.926928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:45.926928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:45.926928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:45.964084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb320177-68cb-4507-980b-5013804f5cdb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:46.092873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:46.097675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:46.187523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:46.188360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:46.188638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:46.188937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] PATH_INFO: `/v3/users/d5f86863da7d4aa39e65f884774c820d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:46.195706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f86863da7d4aa39e65f884774c820d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:46.198828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:46.273099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 864/1710] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:14:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:46.274481 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:46.277451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:46.318894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583bcba-692e-47b3-b9f4-1b19a2998c35 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:46.323767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:46.325640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:46.325640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:46.325640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] PATH_INFO: `/v3/projects/332a6aeb4ada4c7985045bcf95f38ed5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:46.326794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=332a6aeb4ada4c7985045bcf95f38ed5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:46.329287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:46.577475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f86863da7d4aa39e65f884774c820d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.579441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f86863da7d4aa39e65f884774c820d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.581430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f86863da7d4aa39e65f884774c820d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.589060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332a6aeb4ada4c7985045bcf95f38ed5', 'request_id': 'req-fe1cc05e-f615-442e-b1e1-1c550502f252'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.604208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f86863da7d4aa39e65f884774c820d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.643158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f86863da7d4aa39e65f884774c820d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.715214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332a6aeb4ada4c7985045bcf95f38ed5', 'request_id': 'req-fe1cc05e-f615-442e-b1e1-1c550502f252'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.728589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332a6aeb4ada4c7985045bcf95f38ed5', 'request_id': 'req-fe1cc05e-f615-442e-b1e1-1c550502f252'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.754676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] The token cache is being invalidate because project 332a6aeb4ada4c7985045bcf95f38ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:46.755305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.756167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe1cc05e-f615-442e-b1e1-1c550502f252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:46.762259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 865/1711] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:46 2022] DELETE /identity/v3/projects/332a6aeb4ada4c7985045bcf95f38ed5 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:46.764593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:46.765021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:46.765488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:46.801612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:46.803363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:46.807728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 847/1712] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:45 2022] DELETE /identity/v3/users/d5f86863da7d4aa39e65f884774c820d => generated 0 bytes in 1084 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:46.809249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:46.812693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:46.917351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:46.960755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540c2d8-0217-4459-b69e-b79d4fd8a838 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:46.961176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 866/1713] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:46.963465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:46.972053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:46.976421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:46.979589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:46.981468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:46.981859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] PATH_INFO: `/v3/users/f7ec5b177414433a992a163a9dbc5979` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:46.985949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ec5b177414433a992a163a9dbc5979)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:47.004801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:47.192091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9bc2c-25dd-4f72-9e12-7d146f3449a5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:47.207136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:47.220185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.220637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.220929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.228342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.315888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ec5b177414433a992a163a9dbc5979'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:47.327687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ec5b177414433a992a163a9dbc5979'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:47.327687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ec5b177414433a992a163a9dbc5979'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:47.362259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ec5b177414433a992a163a9dbc5979'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:47.363577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:47.365272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26925d69-9063-46f7-8214-a924c93750c4 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:47.384355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:47.389370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 867/1714] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:14:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:14:47.391597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9af6-29c1-493b-ba85-d8435c4f7a44 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.392554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9af6-29c1-493b-ba85-d8435c4f7a44 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.393116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9af6-29c1-493b-ba85-d8435c4f7a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.425024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fa83bedc-b8e9-4296-8ced-08c8641d0ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ec5b177414433a992a163a9dbc5979'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:47.430762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-576b9af6-29c1-493b-ba85-d8435c4f7a44 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:47.430762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-576b9af6-29c1-493b-ba85-d8435c4f7a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.442117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 868/1715] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:47.444593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee1c28-95b8-48ad-b725-a4b54c6cb989 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.444993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee1c28-95b8-48ad-b725-a4b54c6cb989 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.445373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee1c28-95b8-48ad-b725-a4b54c6cb989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.478859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6dee1c28-95b8-48ad-b725-a4b54c6cb989 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:47.480522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6dee1c28-95b8-48ad-b725-a4b54c6cb989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.512016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 869/1716] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:47.517470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.517966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.518345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.569785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 848/1717] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:46 2022] DELETE /identity/v3/users/f7ec5b177414433a992a163a9dbc5979 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:47.571927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc8b69-d066-4851-897b-491715e357b1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.572578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc8b69-d066-4851-897b-491715e357b1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.572993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc8b69-d066-4851-897b-491715e357b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.604197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-62fc8b69-d066-4851-897b-491715e357b1 None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:47.605669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:47.606322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-62fc8b69-d066-4851-897b-491715e357b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.607471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.689155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 849/1718] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Sep 7 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:47.692894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.694710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.696758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.760583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:47.767667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.852838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 870/1719] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:47.853922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:47.859348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.888349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129b02-9d7d-426d-8299-e5468f875b85 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:47.906014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:47.907498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:47.908433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:47.909507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:47.932631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 850/1720] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:47.933537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:47.938820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:47.940775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:47.946471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:47.976507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed81db-5c2d-4b2c-bb1a-65d1b75a6964 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:47.990117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 871/1721] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:47.991366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.004538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.005040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.005508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.006020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.016584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.018402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.021624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43820a7-d450-4196-aa3f-26ff7edfd030 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.037621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.039853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.041338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.043862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.045933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:48.046254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.053865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 851/1722] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:48.054707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.062011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.090867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 872/1723] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:48.091053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db9a93-f55b-41f7-b50d-30592065da18 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.092779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.105035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.105035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.105035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.105035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.105509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:48.105509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.109822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.124639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89047f5c-e69f-4d62-a927-d0d1ba57cd52 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.133433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.134326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.134892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.135355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.136367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.136832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 852/1724] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:48.140332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.148835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.185142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7f6a6-f1e9-44cc-9592-e4db33e86b3b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.192117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.193397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.193755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.194099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.194978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.197481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.303894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 853/1725] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:48 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:48.307937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.310966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 873/1726] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:48 2022] POST /identity/v3/projects => generated 405 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:48.313237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.577849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051c6b2-8a93-414a-a900-d943f47d7b0c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.744662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.746911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cff21e-348f-4f3b-8ed8-88799f1a2030 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.747413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.747857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.748267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.749732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.753280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.790210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.794713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.795207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.795688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.796578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.798859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.892065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 874/1727] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:48 2022] POST /identity/v3/users => generated 303 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:48.896393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.909341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4ee7e-c773-402d-83f8-fe0b56c881be admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.917790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.920601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.920601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.920601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.920601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.921940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:48.940366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 875/1728] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:14:48.941398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.950823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:48.950965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 854/1729] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:48 2022] POST /identity/v3/users => generated 294 bytes in 643 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:48.955382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:48.981381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fd982-12ae-40e9-9535-9d02fe250c1e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.984097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc911a-362f-4391-b331-24bf1391f9a5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:48.993656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.994366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:48.994685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:48.995986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:48.996874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:48.999448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:48.999734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.000462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.002840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.003858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:49.004550 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:49.008009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:49.026363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 855/1730] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:49.037790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:49.042693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 876/1731] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:49.047630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:49.048237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:49.065049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b94bc-9f29-4e42-a3ea-2e16a2908f7d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:49.067077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65fe4-4748-41e9-ac23-9a5e2d827d31 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:49.072225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:49.073114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.078246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbf267-435f-4fef-b2ae-c7be2361e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:49.079281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbf267-435f-4fef-b2ae-c7be2361e492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.081060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbf267-435f-4fef-b2ae-c7be2361e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.081421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbf267-435f-4fef-b2ae-c7be2361e492 admin admin] PATH_INFO: `/v3/projects/327e7ac5884b4467bda0fc95f8a53f14/users/f228905356d14ca5bfcb8e8186985287/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.087011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.087390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.093176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:49.095304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:49.111534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 856/1732] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:49.115813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:49.119529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbf267-435f-4fef-b2ae-c7be2361e492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327e7ac5884b4467bda0fc95f8a53f14, user_id=f228905356d14ca5bfcb8e8186985287, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:49.125198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbf267-435f-4fef-b2ae-c7be2361e492 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:49.130599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc4c32-9d8a-4ee6-87e5-857a998af372 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:49.135483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35ed21-1616-4511-9c96-7757fbd46a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:49.136510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35ed21-1616-4511-9c96-7757fbd46a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.137077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35ed21-1616-4511-9c96-7757fbd46a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.137843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35ed21-1616-4511-9c96-7757fbd46a32 admin admin] PATH_INFO: `/v3/projects/a8aeafd059ed4c9e9c5fcaa4e5323742/users/cb0974ddcc5c463db9a2f4ecd72f7add/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.161244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35ed21-1616-4511-9c96-7757fbd46a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8aeafd059ed4c9e9c5fcaa4e5323742, user_id=cb0974ddcc5c463db9a2f4ecd72f7add, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:49.165532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35ed21-1616-4511-9c96-7757fbd46a32 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:49.258497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 877/1733] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:49 2022] PUT /identity/v3/projects/327e7ac5884b4467bda0fc95f8a53f14/users/f228905356d14ca5bfcb8e8186985287/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:49.273438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.273739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.274023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.319954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] MFA Rules not processed for user `f228905356d14ca5bfcb8e8186985287`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:49.322853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:49.440978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 857/1734] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:49 2022] PUT /identity/v3/projects/a8aeafd059ed4c9e9c5fcaa4e5323742/users/cb0974ddcc5c463db9a2f4ecd72f7add/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:49.455108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.455562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.455849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.476164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 878/1735] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:49.484121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:49.564720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] MFA Rules not processed for user `cb0974ddcc5c463db9a2f4ecd72f7add`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:49.569820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:49.621297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de810a-f559-4fed-b0ad-7b65c68f8aa5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:49.749668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 858/1736] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:14:49.754346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:49.830411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:49.831053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:49.831337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:49.831611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:49.835366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:49.837988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.019068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb0235-6592-4552-93b2-dafa6e505936 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.030847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.031382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.031766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.032111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.033322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.034648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 879/1737] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:49 2022] POST /identity/v3/projects => generated 403 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:50.036390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.037612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:50.042450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:50.185778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 859/1738] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:49 2022] POST /identity/v3/projects => generated 385 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:50.185778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:50.269332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420795-b894-444c-a6ab-b039e737def0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.353599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c2cc1-42aa-4dd1-bfa9-64341c50e39b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.367310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.369372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.369662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.371817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.373795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.376439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.474977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 860/1739] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:50 2022] POST /identity/v3/users => generated 308 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:50.477378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:50.486519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.487503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.487888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.488264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] PATH_INFO: `/v3/projects/91b7336a644a46ff9a0c5b55a16169b9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.489722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b7336a644a46ff9a0c5b55a16169b9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.491800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.496292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:50.505442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c487-ce02-43b3-8aea-4a4700cdaedd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.519321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.520258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.520702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.521320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.522138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.527308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.648946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:50.699502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 861/1740] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:50 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:50.699502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:50.699502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:50.716833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e014d-5482-4793-841e-cbf08675d724 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.718326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b7336a644a46ff9a0c5b55a16169b9', 'request_id': 'req-dab97c30-6fda-49ff-a12d-0d9c1506d360'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:50.800398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b7336a644a46ff9a0c5b55a16169b9', 'request_id': 'req-dab97c30-6fda-49ff-a12d-0d9c1506d360'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:50.805954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.807945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.808949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.809249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] PATH_INFO: `/v3/projects/4c8e1d524f654d1a8e52fd54b029e3ca` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.809451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b7336a644a46ff9a0c5b55a16169b9', 'request_id': 'req-dab97c30-6fda-49ff-a12d-0d9c1506d360'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:50.810727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c8e1d524f654d1a8e52fd54b029e3ca)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.817087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.836782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] The token cache is being invalidate because project 91b7336a644a46ff9a0c5b55a16169b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:50.838120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:50.839951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:50.843324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 880/1741] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:50 2022] DELETE /identity/v3/projects/91b7336a644a46ff9a0c5b55a16169b9 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:50.848447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:50.855850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:50.880774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab97c30-6fda-49ff-a12d-0d9c1506d360 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.893718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.894543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.894928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.895986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.897738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.902445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:50.923483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 881/1742] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:50.925029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:50.928913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:50.954072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9c255-6a42-42fa-8d3f-83e69e40c0da admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:50.963361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a6732-b419-4c74-bddf-42412f8b9025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:50.964473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a6732-b419-4c74-bddf-42412f8b9025 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:50.964807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a6732-b419-4c74-bddf-42412f8b9025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:50.965151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a6732-b419-4c74-bddf-42412f8b9025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:50.967750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a6732-b419-4c74-bddf-42412f8b9025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:50.972185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a6732-b419-4c74-bddf-42412f8b9025 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:51.038834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 882/1743] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:51.043734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.043734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.043734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.044265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c8e1d524f654d1a8e52fd54b029e3ca', 'request_id': 'req-a53d0455-83f7-4ce5-a72d-4d74e4351cca'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:51.071291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c8e1d524f654d1a8e52fd54b029e3ca', 'request_id': 'req-a53d0455-83f7-4ce5-a72d-4d74e4351cca'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:51.101897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c8e1d524f654d1a8e52fd54b029e3ca', 'request_id': 'req-a53d0455-83f7-4ce5-a72d-4d74e4351cca'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:51.104055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:51.109059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.209614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] The token cache is being invalidate because project 4c8e1d524f654d1a8e52fd54b029e3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:51.210197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:51.221257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:51.225526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 862/1744] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:50 2022] DELETE /identity/v3/projects/4c8e1d524f654d1a8e52fd54b029e3ca => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:51.226825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:51.231351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.247433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 883/1745] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:51.248286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:51.267124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.298606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709f41-6de4-4e7c-be76-cfdfbcf7dca5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:51.306292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f1d0-f841-441e-b963-734932914556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:51.307439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f1d0-f841-441e-b963-734932914556 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.307439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f1d0-f841-441e-b963-734932914556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.307740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f1d0-f841-441e-b963-734932914556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.308646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f1d0-f841-441e-b963-734932914556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:51.310191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f1d0-f841-441e-b963-734932914556 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:51.352666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 884/1746] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:51.362146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4065c1-4aaf-4dcf-8348-c34211f1b7fe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.363535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4065c1-4aaf-4dcf-8348-c34211f1b7fe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.363535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4065c1-4aaf-4dcf-8348-c34211f1b7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.379173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d0455-83f7-4ce5-a72d-4d74e4351cca admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:51.386517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b416c9-d61f-44d0-ad94-dfe30c698e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:51.387600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b416c9-d61f-44d0-ad94-dfe30c698e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.388082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b416c9-d61f-44d0-ad94-dfe30c698e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.388194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:51.388673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b416c9-d61f-44d0-ad94-dfe30c698e84 admin admin] PATH_INFO: `/v3/projects/e60501d5afd240c09b256f43e29f76c3/users/fe25c1b0ac7d4718a2e73d127ddf5925/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.428561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1f4065c1-4aaf-4dcf-8348-c34211f1b7fe None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:51.430080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1f4065c1-4aaf-4dcf-8348-c34211f1b7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.435525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b416c9-d61f-44d0-ad94-dfe30c698e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60501d5afd240c09b256f43e29f76c3, user_id=fe25c1b0ac7d4718a2e73d127ddf5925, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:51.443454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b416c9-d61f-44d0-ad94-dfe30c698e84 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:51.458598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 885/1747] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:51.458598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.458598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.459228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.498096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:51.500407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.512710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 886/1748] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:51.513298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:51.533814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6cb7-8f76-4c83-abed-9f23ba5f2ba0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:51.541796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:51.543102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.543415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.543948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] PATH_INFO: `/v3/projects/3efa775b67604802b6a64961d8a77dab/users/e727fbd24c5f48f398cafdd0eb585863/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.568049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efa775b67604802b6a64961d8a77dab, user_id=e727fbd24c5f48f398cafdd0eb585863, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:51.575571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:51.581256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 863/1749] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:51 2022] PUT /identity/v3/projects/e60501d5afd240c09b256f43e29f76c3/users/fe25c1b0ac7d4718a2e73d127ddf5925/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:51.582808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.583934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.584425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.627547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:51.628632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.662493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 887/1750] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:51 2022] PUT /identity/v3/projects/3efa775b67604802b6a64961d8a77dab/users/e727fbd24c5f48f398cafdd0eb585863/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:51.664319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:51.832740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 864/1751] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:51.860368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:51.890170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ee63-13bc-4f34-80c0-e16bbd83f473 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:51.910585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a396-c78f-424c-ae28-da1033a9105b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:51.927761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a87333-9bc5-415b-a63c-59d23ccd29b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:51.928449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a87333-9bc5-415b-a63c-59d23ccd29b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.928728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a87333-9bc5-415b-a63c-59d23ccd29b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.929001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a87333-9bc5-415b-a63c-59d23ccd29b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.929666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a87333-9bc5-415b-a63c-59d23ccd29b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:51.931348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:51.931518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a87333-9bc5-415b-a63c-59d23ccd29b1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:51.933160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.934207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.934589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.937858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:51.938775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:51.945562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 865/1752] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:51.959483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:51.959483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:51.959483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:51.967929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:51.982958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] MFA Rules not processed for user `e727fbd24c5f48f398cafdd0eb585863`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:51.985493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:51.997043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 888/1753] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:51.999059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.023759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:52.040623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207403e2-f461-4629-aaa2-b60600b7280b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:52.061464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:52.061464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.061464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.061464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.061912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:14:52.061912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:52.071022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:52.104612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 889/1754] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:14:52.105827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.137130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e602-6a6f-4784-9d0d-d114b6ac27b7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:52.167857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 866/1755] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:52.168354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:52.168830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.169364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.169659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] PATH_INFO: `/v3/projects/e60501d5afd240c09b256f43e29f76c3/users/fe25c1b0ac7d4718a2e73d127ddf5925/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.172906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.173627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60501d5afd240c09b256f43e29f76c3, user_id=fe25c1b0ac7d4718a2e73d127ddf5925, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:52.179488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:52.209692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:52.224508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 890/1756] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:52 2022] PUT /identity/v3/projects/e60501d5afd240c09b256f43e29f76c3/users/fe25c1b0ac7d4718a2e73d127ddf5925/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:52.240421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.251534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1c90f-cdcb-4954-8fea-291aacaa87fe None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:52.401968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3047dc9-2320-48da-a275-d5746f643c41 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:52.412073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57952cac-a9c1-4c55-a651-39b6dc55f715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:52.414151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57952cac-a9c1-4c55-a651-39b6dc55f715 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.414362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57952cac-a9c1-4c55-a651-39b6dc55f715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.414667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57952cac-a9c1-4c55-a651-39b6dc55f715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.417990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57952cac-a9c1-4c55-a651-39b6dc55f715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:52.426229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57952cac-a9c1-4c55-a651-39b6dc55f715 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:52.449899 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:52.452884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.453168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.453444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.456178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:52.459728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:52.597728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 867/1757] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:52 2022] POST /identity/v3/projects => generated 381 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:52.597728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.597982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 891/1758] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:52 2022] POST /identity/v3/projects => generated 379 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:52.597982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.598215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.598965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.751394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] MFA Rules not processed for user `fe25c1b0ac7d4718a2e73d127ddf5925`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:52.756051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:52.767619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cc7d0-4373-47b7-a098-dcb89cce61da admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:52.779193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b253-e3b4-40bf-a7cd-09f58044e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:52.783781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b253-e3b4-40bf-a7cd-09f58044e502 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.783781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b253-e3b4-40bf-a7cd-09f58044e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.783781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b253-e3b4-40bf-a7cd-09f58044e502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.783781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b253-e3b4-40bf-a7cd-09f58044e502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:52.783781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b253-e3b4-40bf-a7cd-09f58044e502 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:52.938500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 868/1759] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:52 2022] POST /identity/v3/projects => generated 387 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:52.939303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a757a-bc41-41e1-9834-df48023b7d04 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:52.943963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a757a-bc41-41e1-9834-df48023b7d04 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:52.945311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a757a-bc41-41e1-9834-df48023b7d04 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:52.949699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 869/1760] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:14:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:52.952073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 892/1761] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:52.952591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a757a-bc41-41e1-9834-df48023b7d04 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.953019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:52.975906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbb19b-7859-42d6-9042-45c8ce41c15a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.113018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:INSERT=7,keystone:DELETE=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:53.142129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:53.185722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a757a-bc41-41e1-9834-df48023b7d04 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.204595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.206578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.206578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.207437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe343350-37a2-43fb-b331-f2f011534f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.209935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.214937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe343350-37a2-43fb-b331-f2f011534f1f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.215257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe343350-37a2-43fb-b331-f2f011534f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.216224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe343350-37a2-43fb-b331-f2f011534f1f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.217721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe343350-37a2-43fb-b331-f2f011534f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.219129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.219766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe343350-37a2-43fb-b331-f2f011534f1f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.221535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.330945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 893/1762] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:52 2022] POST /identity/v3/users => generated 291 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:53.332940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c821e6-2f84-474a-900a-525317c06eb1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.333200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c821e6-2f84-474a-900a-525317c06eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.333700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c821e6-2f84-474a-900a-525317c06eb1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.335681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 894/1763] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:14:53.336863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c821e6-2f84-474a-900a-525317c06eb1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.340553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 870/1764] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:52 2022] POST /identity/v3/users => generated 290 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:53.342814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.345013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:53.354696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c821e6-2f84-474a-900a-525317c06eb1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.360467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb823c5-2bf3-470d-8ec4-c033a7dcf950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.361619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb823c5-2bf3-470d-8ec4-c033a7dcf950 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.361973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb823c5-2bf3-470d-8ec4-c033a7dcf950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.363135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb823c5-2bf3-470d-8ec4-c033a7dcf950 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.369800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb823c5-2bf3-470d-8ec4-c033a7dcf950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.372078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb823c5-2bf3-470d-8ec4-c033a7dcf950 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.456542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94520028-ed99-48e7-9625-5a95fcd152b7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.461028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.461959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.462663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.462940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.481134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 895/1765] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:53 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:53.481134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660598-335f-426c-845c-295823997e68 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.481134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660598-335f-426c-845c-295823997e68 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.481134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660598-335f-426c-845c-295823997e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.540222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aa660598-335f-426c-845c-295823997e68 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:53.546459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aa660598-335f-426c-845c-295823997e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:53.558793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.560431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.573865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 871/1766] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:53 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:53.574845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.588003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a61335-b926-49a9-9e71-38089ac1cb78 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.596204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.596891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.597176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.597871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.598468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.600684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.628571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 872/1767] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:53.628571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.647522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8fcd5-86bb-471a-9db1-7f186d76634b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.657914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c960af-432c-46d3-97d3-e3e066ad4e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.659742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c960af-432c-46d3-97d3-e3e066ad4e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.660130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c960af-432c-46d3-97d3-e3e066ad4e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.660600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c960af-432c-46d3-97d3-e3e066ad4e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.661672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c960af-432c-46d3-97d3-e3e066ad4e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.663709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c960af-432c-46d3-97d3-e3e066ad4e43 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.681801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 873/1768] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:53.684346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.684760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.685114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.767606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:53.772783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:53.792630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 896/1769] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:14:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:53.802235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa660598-335f-426c-845c-295823997e68 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.816985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa660598-335f-426c-845c-295823997e68 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.822675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 874/1770] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:14:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:53.829093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.829667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.830358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.834519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.834519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.834519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.834519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.862884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 897/1771] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:53.866233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.887911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b28d15-c723-4d67-a2bf-9fcdda8a25dc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.890800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acbbe5-3f4e-454b-91fd-732d5068e927 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.902298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.902514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:53.904494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.905633 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.906335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.906726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:53.907033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:53.907486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:53.908053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.908971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:53.914076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.923078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:53.948635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 875/1772] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:53.950064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.953081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 898/1773] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:53.956020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:53.984884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d0144-70d4-43b3-94cf-9e7509346b93 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:53.986158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6a878-ac96-4354-b28f-824278df807f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:54.002244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:54.003426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:54.004274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:54.004764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] PATH_INFO: `/v3/users/f228905356d14ca5bfcb8e8186985287` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:54.006382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f228905356d14ca5bfcb8e8186985287)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:54.008877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:54.015229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:54.017250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:54.017933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:54.018212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:54.018490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] PATH_INFO: `/v3/projects/6e4662b399604da29439da207cc5de68/users/1ab7accda2bf42439f155cea70debdc8/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:54.049050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:54.084742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4662b399604da29439da207cc5de68, user_id=1ab7accda2bf42439f155cea70debdc8, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:54.093010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:54.219824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 876/1774] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:53 2022] PUT /identity/v3/projects/6e4662b399604da29439da207cc5de68/users/1ab7accda2bf42439f155cea70debdc8/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:54.219824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:54.249802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f228905356d14ca5bfcb8e8186985287'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:54.251341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f228905356d14ca5bfcb8e8186985287'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:54.252211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f228905356d14ca5bfcb8e8186985287'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:54.261742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f228905356d14ca5bfcb8e8186985287'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:54.297743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f228905356d14ca5bfcb8e8186985287'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:54.386065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 899/1775] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:53 2022] DELETE /identity/v3/users/f228905356d14ca5bfcb8e8186985287 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:54.390814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:54.396244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:54.401889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cd6f4-6255-4258-bab6-a220abc65eb5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:54.541715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fed300-a528-412d-88b3-ebfa6e88fb72 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:54.554019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:54.555083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:54.563726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:54.563726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] PATH_INFO: `/v3/projects/0c4e175becee44cf9e17422c0ed1b99a/users/20f336c782624850aa0864c99132fbd9/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:54.577530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4e175becee44cf9e17422c0ed1b99a, user_id=20f336c782624850aa0864c99132fbd9, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:54.581077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:54.585569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:54.586263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:54.586521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:54.586924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] PATH_INFO: `/v3/projects/d93d3a908d324b9ca5a05f438146ef0f/users/08c7f501d2be466d8a83033d0e73ef69/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:54.619313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93d3a908d324b9ca5a05f438146ef0f, user_id=08c7f501d2be466d8a83033d0e73ef69, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:54.624870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:54.651199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:54.659513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 900/1776] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:54 2022] PUT /identity/v3/projects/0c4e175becee44cf9e17422c0ed1b99a/users/20f336c782624850aa0864c99132fbd9/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:54.659513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:54.665361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:54.713286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 877/1777] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:54 2022] PUT /identity/v3/projects/d93d3a908d324b9ca5a05f438146ef0f/users/08c7f501d2be466d8a83033d0e73ef69/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:54.714253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:54.726380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:54.838377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6bca4-b5cb-467f-8d47-5e7179904511 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:54.871994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd02930-596e-4e6e-8169-7f5c55529e98 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:54.887917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:54.888635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:54.889227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:54.889533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:54.890268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:54.894536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:54.928920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 878/1778] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:54.929895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:54.932115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.042257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:55.079042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:55.081439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.081804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.082009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.089875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.108207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a869466-26a5-4e02-bb59-f10acefc709d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:55.115097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:55.115906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.123495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.123495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.123784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:55.123868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.123940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e62ff4-e4fc-462e-bce6-6e698884ab7a service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:55.131510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 901/1779] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:55.132855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e2e8-7d3a-4a96-9633-c79b513eb5b5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.133154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e2e8-7d3a-4a96-9633-c79b513eb5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.133397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e2e8-7d3a-4a96-9633-c79b513eb5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.134516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:55.137592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449f7e4-48f8-4d80-a544-3900ffcd8485 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:55.163326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ab83e2e8-7d3a-4a96-9633-c79b513eb5b5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:55.168033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 879/1780] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:55.169286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab83e2e8-7d3a-4a96-9633-c79b513eb5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.171081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.171366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.171639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.191123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 902/1781] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:55.192609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.192897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.193174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.227772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] MFA Rules not processed for user `20f336c782624850aa0864c99132fbd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:55.228985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.248391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] MFA Rules not processed for user `08c7f501d2be466d8a83033d0e73ef69`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:55.253565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.427543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 880/1782] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:55.427797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:55.439047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 903/1783] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:55.440035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:55.441594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.448461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06687c74-5822-4785-a16d-6ae27e0eea0c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:55.456552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:55.459907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.459907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.459907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] PATH_INFO: `/v3/projects/6e4662b399604da29439da207cc5de68/users/1ab7accda2bf42439f155cea70debdc8/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.478842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4662b399604da29439da207cc5de68, user_id=1ab7accda2bf42439f155cea70debdc8, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:55.481833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:55.494588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 881/1784] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:55 2022] PUT /identity/v3/projects/6e4662b399604da29439da207cc5de68/users/1ab7accda2bf42439f155cea70debdc8/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:55.495283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:55.534708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f91682-95d4-43fb-bf29-363402a62d8b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:55.637263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:55.637967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.638317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.639067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.649093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1e0a0-2b65-4488-8170-5756581953bc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:55.658928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:55.661194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:55.665341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cecb2-ef81-491e-88ba-8a5952a20199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:55.666051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cecb2-ef81-491e-88ba-8a5952a20199 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.672347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cecb2-ef81-491e-88ba-8a5952a20199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.672347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cecb2-ef81-491e-88ba-8a5952a20199 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.672347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cecb2-ef81-491e-88ba-8a5952a20199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:55.672615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cecb2-ef81-491e-88ba-8a5952a20199 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:55.692145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 904/1785] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:55.695245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:55.702494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:55.719787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae9aa8-d697-4381-8d91-d468521375cb service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:55.736108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:55.737784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.737784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.737784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.740575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:55.742738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:55.836838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 882/1786] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/projects => generated 381 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:55.838372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:55.838816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:55.839200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:55.872062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 905/1787] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:55.872062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:55.910056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] MFA Rules not processed for user `1ab7accda2bf42439f155cea70debdc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:55.911369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:56.056871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782e1a2-ecee-4f4e-b9a7-402d576d18f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.069320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.070082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.070436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.070789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.072264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.075480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.122389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 883/1788] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:56.133864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.146756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c148de4-a6cd-47b3-828b-25f678f84d2d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.158307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.159267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.159741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.160184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.161008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.163167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.177184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 906/1789] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:55 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:56.178085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.181972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:56.262804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 884/1790] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:14:56 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:14:56.262804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.286967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabddcf-d8b1-4de9-ba7c-ec65d626dae5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.306537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.307509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.307749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.308007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.308658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.321010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.328767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4fdd9-d800-4224-a846-bf85dca091f9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.334966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.335984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.336440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.336731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.346353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 885/1791] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:56.350388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.352171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.354397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.369695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02a27b-3dba-40b0-81e9-aba58165bbf8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.373165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 907/1792] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:56.374371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.391622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18333509-58d0-4656-844b-a1b6faea5af0 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.392971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.395192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.395543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.395951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.400945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.400945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.401075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.401075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.401075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.401075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] PATH_INFO: `/v3/projects/0b2802bdfbc0465090621102aa9def20/users/f0d6351b8d5d4e31ada2cd40c63db8aa/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.424076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:56.433735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 886/1793] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:56.437628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.463356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2802bdfbc0465090621102aa9def20, user_id=f0d6351b8d5d4e31ada2cd40c63db8aa, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.466040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.475104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15417f-9360-4b25-9e0a-5d480e9b6c7e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.485261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:56.499778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.500703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.501171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.501652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] PATH_INFO: `/v3/projects/5d3220aed0de4a52b3fcb0c8b90e8d8d/users/9e75a0236ded483e826a0520e7ecc315/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.526219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3220aed0de4a52b3fcb0c8b90e8d8d, user_id=9e75a0236ded483e826a0520e7ecc315, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.529546 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.568186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 908/1794] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:56 2022] PUT /identity/v3/projects/0b2802bdfbc0465090621102aa9def20/users/f0d6351b8d5d4e31ada2cd40c63db8aa/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:56.572730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.621989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 887/1795] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:56 2022] PUT /identity/v3/projects/5d3220aed0de4a52b3fcb0c8b90e8d8d/users/9e75a0236ded483e826a0520e7ecc315/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:56.626134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.774698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e775c3b-e10c-4599-9d8c-7f2e9a7d221d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.812984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d692c34-2f7f-445f-a211-a6bfe7e3fd3d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.815843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.816502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.816832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.817153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.817774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.832970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.834165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.835755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.835947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.836231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.839978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.847546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.879958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 888/1796] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:56.880785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.906932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 909/1797] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:14:56.906932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:56.907184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b51a2d-448b-4a64-aea1-ba0ca10accf1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.907689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e993fc9-03d5-4d3a-bbfd-3dd9439aa5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:56.908430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993fc9-03d5-4d3a-bbfd-3dd9439aa5af admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.908854 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993fc9-03d5-4d3a-bbfd-3dd9439aa5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.909018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993fc9-03d5-4d3a-bbfd-3dd9439aa5af admin admin] PATH_INFO: `/v3/projects/5d3220aed0de4a52b3fcb0c8b90e8d8d/users/9e75a0236ded483e826a0520e7ecc315/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.911471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e993fc9-03d5-4d3a-bbfd-3dd9439aa5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3220aed0de4a52b3fcb0c8b90e8d8d, user_id=9e75a0236ded483e826a0520e7ecc315, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:56.914328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e993fc9-03d5-4d3a-bbfd-3dd9439aa5af admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:56.935684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 889/1798] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:56 2022] PUT /identity/v3/projects/5d3220aed0de4a52b3fcb0c8b90e8d8d/users/9e75a0236ded483e826a0520e7ecc315/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:56.937366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:56.937660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:56.937980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:56.941041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300933a-ad77-49ae-b913-4452ca76a701 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:56.989472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] MFA Rules not processed for user `9e75a0236ded483e826a0520e7ecc315`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:56.993729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:57.059174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806ad1d-4c2d-44b6-9860-dce788eb4661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:57.059934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ad1d-4c2d-44b6-9860-dce788eb4661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:57.060224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ad1d-4c2d-44b6-9860-dce788eb4661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:57.061104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ad1d-4c2d-44b6-9860-dce788eb4661 admin admin] PATH_INFO: `/v3/projects/0b2802bdfbc0465090621102aa9def20/users/f0d6351b8d5d4e31ada2cd40c63db8aa/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:57.064437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ad1d-4c2d-44b6-9860-dce788eb4661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2802bdfbc0465090621102aa9def20, user_id=f0d6351b8d5d4e31ada2cd40c63db8aa, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:57.068380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ad1d-4c2d-44b6-9860-dce788eb4661 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:57.098594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 910/1799] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:14:56 2022] PUT /identity/v3/projects/0b2802bdfbc0465090621102aa9def20/users/f0d6351b8d5d4e31ada2cd40c63db8aa/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:57.109919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:57.110581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:57.111114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:57.183204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] MFA Rules not processed for user `f0d6351b8d5d4e31ada2cd40c63db8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:57.185081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:57.289367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 890/1800] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:57.378903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 911/1801] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:14:57.382157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:57.383555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:57.425469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:57.462770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768c801-8d52-4969-bbf6-cb23c0781b6a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:57.546149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:57.546876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:57.550940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:57.556295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:57.643594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:57.645435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:57.732903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:57.734381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:57.758942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 891/1802] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:57.814077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929808a2-ce7b-4066-988b-f096ec5c523e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:57.819463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:57.820321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:57.820499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:57.820893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] PATH_INFO: `/v3/projects/327e7ac5884b4467bda0fc95f8a53f14` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:57.822832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327e7ac5884b4467bda0fc95f8a53f14)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:57.824953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:58.004159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327e7ac5884b4467bda0fc95f8a53f14', 'request_id': 'req-92e11fb4-641a-4377-a8ca-57d0cdba6437'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.041941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:58.053549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95182d-b183-4f28-aa9a-6388b6edbf34 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:58.109080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327e7ac5884b4467bda0fc95f8a53f14', 'request_id': 'req-92e11fb4-641a-4377-a8ca-57d0cdba6437'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.126264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327e7ac5884b4467bda0fc95f8a53f14', 'request_id': 'req-92e11fb4-641a-4377-a8ca-57d0cdba6437'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.143478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:58.145446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] The token cache is being invalidate because project 327e7ac5884b4467bda0fc95f8a53f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:58.145746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.146535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:58.146904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:58.147185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:58.148090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.150490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 912/1803] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:57 2022] DELETE /identity/v3/projects/327e7ac5884b4467bda0fc95f8a53f14 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:14:58.155992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:58.160559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:58.227515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:58.229012 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:58.249666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 892/1804] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:58.250985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:58.271618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef8612-61da-4204-89b6-4dce1d013b4e service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:58.287107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:58.287861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:58.288138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:58.288462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:58.290593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:58.313676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:58.318725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e11fb4-641a-4377-a8ca-57d0cdba6437 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:58.324292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:58.325178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:58.325435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:58.325720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] PATH_INFO: `/v3/users/fe25c1b0ac7d4718a2e73d127ddf5925` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:58.327062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe25c1b0ac7d4718a2e73d127ddf5925)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:58.329004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:58.379168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:58.380699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:58.396248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 893/1805] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:14:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:14:58.489178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:58.506179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe25c1b0ac7d4718a2e73d127ddf5925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.507284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe25c1b0ac7d4718a2e73d127ddf5925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.508952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe25c1b0ac7d4718a2e73d127ddf5925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.535742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe25c1b0ac7d4718a2e73d127ddf5925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.557609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a89a2e9-bc0b-4720-bffa-995519fcfc02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe25c1b0ac7d4718a2e73d127ddf5925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:58.663394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 913/1806] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:14:58 2022] DELETE /identity/v3/users/fe25c1b0ac7d4718a2e73d127ddf5925 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:58.918143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:59.545983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:14:59.564869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2016f-d71d-478a-8111-095de2f2f532 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:14:59.632960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:14:59.633677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:59.634040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:59.634372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] PATH_INFO: `/v3/projects/e60501d5afd240c09b256f43e29f76c3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:59.635665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60501d5afd240c09b256f43e29f76c3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:14:59.637912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:14:59.694086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:14:59.752780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60501d5afd240c09b256f43e29f76c3', 'request_id': 'req-9919e85f-4153-46e5-8d80-a9f8da209938'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:59.759492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60501d5afd240c09b256f43e29f76c3', 'request_id': 'req-9919e85f-4153-46e5-8d80-a9f8da209938'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:59.765822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60501d5afd240c09b256f43e29f76c3', 'request_id': 'req-9919e85f-4153-46e5-8d80-a9f8da209938'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:59.848952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] The token cache is being invalidate because project e60501d5afd240c09b256f43e29f76c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:14:59.849393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:59.850053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9919e85f-4153-46e5-8d80-a9f8da209938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:14:59.859417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 894/1807] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:14:59 2022] DELETE /identity/v3/projects/e60501d5afd240c09b256f43e29f76c3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:14:59.864002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:59.864566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:59.864655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:59.887684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:14:59.889466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:14:59.974958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 914/1808] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:14:59.981067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:14:59.981374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:14:59.981656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:14:59.981881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:00.009914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:00.011428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:00.021013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 895/1809] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:00.027212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.041411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6954c9d-8434-4cf4-a0d6-c83c06796870 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:00.047699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:00.052983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.052983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.052983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.052983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:00.052983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:00.052983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:00.065814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 915/1810] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:00.069939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.081830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16f98-b0f6-4ba7-977f-1f1e2b8f5d29 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:00.084595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:00.085182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.085453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.085716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.086481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:00.088459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:00.322682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 896/1811] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:00 2022] POST /identity/v3/projects => generated 403 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:00.323079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.337966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4699f4-f9e0-471d-9cbf-b1b08b667b8c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:00.499971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:00.499971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.499971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.499971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.500364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:00.500364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:00.616507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 916/1812] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:00 2022] POST /identity/v3/users => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:00.617200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.628476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd21445-b3c6-467a-9060-ecde66fb5ad8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:00.633165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:00.633842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.638474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.638474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.638474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:00.638474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:00.650824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 897/1813] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:00.652425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.663640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da283f93-9487-41c2-9fcf-4b7d49e3f220 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:00.668292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1570d-f544-4dec-ae5e-2454777cff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:00.668919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1570d-f544-4dec-ae5e-2454777cff06 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.669202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1570d-f544-4dec-ae5e-2454777cff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.669479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1570d-f544-4dec-ae5e-2454777cff06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.670716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1570d-f544-4dec-ae5e-2454777cff06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:00.672296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1570d-f544-4dec-ae5e-2454777cff06 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:00.684370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 917/1814] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:00.687958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.697935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0f21a-2906-45e9-b612-cc4539db9007 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:00.702501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:00.703211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.703493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.703767 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] PATH_INFO: `/v3/projects/be9e5f75760f43f48c6abbc5b37d494b/users/ce63290a444042f1803d695583da452b/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.718831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9e5f75760f43f48c6abbc5b37d494b, user_id=ce63290a444042f1803d695583da452b, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:00.721556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:00.731341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:00.883787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 898/1815] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:00 2022] PUT /identity/v3/projects/be9e5f75760f43f48c6abbc5b37d494b/users/ce63290a444042f1803d695583da452b/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:00.889372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290baee-1008-43bf-b5a5-65c5096a8100 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:00.889372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290baee-1008-43bf-b5a5-65c5096a8100 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:00.889372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290baee-1008-43bf-b5a5-65c5096a8100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:00.909636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7290baee-1008-43bf-b5a5-65c5096a8100 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:00.910969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7290baee-1008-43bf-b5a5-65c5096a8100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:00.971691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 918/1816] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:00.981255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:00.983703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:01.041207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267bf38-8f53-4ac6-a2da-0570198c7382 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:01.045395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:01.045991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:01.046264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:01.046532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:01.055756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:01.057166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:01.060090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:01.067434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 899/1817] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:01.191418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:02.418414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:02.418924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:02.419310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:02.453388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] MFA Rules not processed for user `ce63290a444042f1803d695583da452b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:02.454862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:02.483507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:02.548719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 919/1818] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:02.553320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:02.565686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f50617-7f41-4718-a98f-ef129b11d9d5 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:02.570653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:02.571339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:02.571614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:02.571946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:02.572578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:02.574469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:02.675627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 900/1819] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:02 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:02.679871 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:02.691222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bbe96-584d-412a-8201-0864610f428f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:02.747983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:02.748594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:02.748878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:02.749155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:02.749789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:02.751737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:02.832794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 920/1820] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:02 2022] POST /identity/v3/users => generated 309 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:02.836939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:02.848146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54613-9e9a-42d2-a9c1-cd97b123dac0 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:02.853133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:02.853777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:02.854055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:02.854356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:02.855038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:02.857257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:02.868160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 901/1821] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:02.872219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:02.885943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b0c5-0fdc-44a8-bcfa-e3bba1757d44 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:02.891182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:02.891852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:02.892127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:02.892399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] PATH_INFO: `/v3/projects/069ed2a653724324802a89b795266282/users/5376a895d2dc406fbb25132a0f07f71e/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:02.908590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=069ed2a653724324802a89b795266282, user_id=5376a895d2dc406fbb25132a0f07f71e, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:02.911321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:03.004011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 921/1822] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:02 2022] PUT /identity/v3/projects/069ed2a653724324802a89b795266282/users/5376a895d2dc406fbb25132a0f07f71e/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:03.007883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:03.015075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:03.019266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71952a6f-d138-4c46-bcca-78e287fc66a9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:03.076324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a4b0b-613e-4c49-9186-8700ea5dc5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:03.077022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a4b0b-613e-4c49-9186-8700ea5dc5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:03.077190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a4b0b-613e-4c49-9186-8700ea5dc5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:03.077464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a4b0b-613e-4c49-9186-8700ea5dc5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:03.078117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a4b0b-613e-4c49-9186-8700ea5dc5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:03.079732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a4b0b-613e-4c49-9186-8700ea5dc5a8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:03.092959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 902/1823] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:03.094499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:03.106818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db7c33-91bb-4781-96ad-6cfec4c4a46c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:03.111697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054f0c3-1701-4231-beec-27a7e37354a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:03.112346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054f0c3-1701-4231-beec-27a7e37354a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:03.112626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054f0c3-1701-4231-beec-27a7e37354a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:03.112898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054f0c3-1701-4231-beec-27a7e37354a5 admin admin] PATH_INFO: `/v3/projects/069ed2a653724324802a89b795266282/users/5376a895d2dc406fbb25132a0f07f71e/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:03.114837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054f0c3-1701-4231-beec-27a7e37354a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069ed2a653724324802a89b795266282, user_id=5376a895d2dc406fbb25132a0f07f71e, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:03.117561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054f0c3-1701-4231-beec-27a7e37354a5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:03.128270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 922/1824] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:03 2022] PUT /identity/v3/projects/069ed2a653724324802a89b795266282/users/5376a895d2dc406fbb25132a0f07f71e/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:04.493725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4348cd-94d3-4c3e-8725-fca0808f4ab5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:04.494208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4348cd-94d3-4c3e-8725-fca0808f4ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:04.494297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4348cd-94d3-4c3e-8725-fca0808f4ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:04.525088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5a4348cd-94d3-4c3e-8725-fca0808f4ab5 None None] MFA Rules not processed for user `5376a895d2dc406fbb25132a0f07f71e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:04.526369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5a4348cd-94d3-4c3e-8725-fca0808f4ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:04.603645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 903/1825] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:05.361610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73c01-d02d-4c0a-a572-bdc646919aa7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:05.362029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73c01-d02d-4c0a-a572-bdc646919aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:05.362273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73c01-d02d-4c0a-a572-bdc646919aa7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:05.363996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 923/1826] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:15:05.396603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:05.396603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:05.396603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:05.421458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:05.423016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:05.498157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 904/1827] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:05.517916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73c01-d02d-4c0a-a572-bdc646919aa7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:05.520358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9cf73c01-d02d-4c0a-a572-bdc646919aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:05.545599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73c01-d02d-4c0a-a572-bdc646919aa7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:05.554360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:05.555145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:05.555529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:05.555934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:05.566001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:05.567858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:05.579125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 924/1828] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:07.113375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:07.120179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:07.122135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:07.164900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:07.227588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6730e3-c92a-47eb-9e88-bb2cbeafe363 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:07.228395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9923d7-3467-4e8c-abee-d1064342a8fd service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:07.236254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:07.236906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:07.237540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:07.237971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:07.238497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] PATH_INFO: `/v3/users/ce63290a444042f1803d695583da452b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:07.239636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:07.240003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:07.240354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:07.240651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce63290a444042f1803d695583da452b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:07.242741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:07.243516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:07.266707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:07.314161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:07.315787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:07.325397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 925/1829] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:07.495232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce63290a444042f1803d695583da452b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:07.495974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce63290a444042f1803d695583da452b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:07.496586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce63290a444042f1803d695583da452b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:07.503884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce63290a444042f1803d695583da452b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:07.516081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce63290a444042f1803d695583da452b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:07.577757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 905/1830] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:07 2022] DELETE /identity/v3/users/ce63290a444042f1803d695583da452b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:07.876012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:07.889368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5d9ec-144a-4d0d-ad84-a1838d62af34 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:08.011728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:08.012965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:08.013730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:08.014197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] PATH_INFO: `/v3/projects/be9e5f75760f43f48c6abbc5b37d494b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:08.017905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9e5f75760f43f48c6abbc5b37d494b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:08.021211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:08.150618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9e5f75760f43f48c6abbc5b37d494b', 'request_id': 'req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.241396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9e5f75760f43f48c6abbc5b37d494b', 'request_id': 'req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.251507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9e5f75760f43f48c6abbc5b37d494b', 'request_id': 'req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.262086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] The token cache is being invalidate because project be9e5f75760f43f48c6abbc5b37d494b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:08.262376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.263104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.265500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 926/1831] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:07 2022] DELETE /identity/v3/projects/be9e5f75760f43f48c6abbc5b37d494b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:08.269601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:08.271286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:08.285911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cfee-8a55-40cd-9f17-4dd4d9ef0530 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:08.345005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:08.345636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:08.349455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:08.349455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] PATH_INFO: `/v3/users/5376a895d2dc406fbb25132a0f07f71e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:08.349455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5376a895d2dc406fbb25132a0f07f71e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:08.349455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:08.574185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:08.576010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5376a895d2dc406fbb25132a0f07f71e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.576648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5376a895d2dc406fbb25132a0f07f71e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.577772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5376a895d2dc406fbb25132a0f07f71e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.584700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5376a895d2dc406fbb25132a0f07f71e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.597532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-608349c6-6043-49cc-aad8-2e580d704f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5376a895d2dc406fbb25132a0f07f71e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:08.704707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 906/1832] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:08 2022] DELETE /identity/v3/users/5376a895d2dc406fbb25132a0f07f71e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:09.125308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.127096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:09.143784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ae0b5-f988-4e60-8ef1-1f0cacff9acf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.193776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:09.207417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.208064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.208417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.208702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] PATH_INFO: `/v3/projects/069ed2a653724324802a89b795266282` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.215280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=069ed2a653724324802a89b795266282)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.217333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:09.317758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069ed2a653724324802a89b795266282', 'request_id': 'req-af00724f-649e-436d-94d7-b39aa5ca3a7c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:09.372440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069ed2a653724324802a89b795266282', 'request_id': 'req-af00724f-649e-436d-94d7-b39aa5ca3a7c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:09.378918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069ed2a653724324802a89b795266282', 'request_id': 'req-af00724f-649e-436d-94d7-b39aa5ca3a7c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:09.389259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] The token cache is being invalidate because project 069ed2a653724324802a89b795266282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:09.389536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:09.390169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-af00724f-649e-436d-94d7-b39aa5ca3a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:09.391672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 927/1833] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:09 2022] DELETE /identity/v3/projects/069ed2a653724324802a89b795266282 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:09.397304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.397763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.397852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.418250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:09.419562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:09.478196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 907/1834] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:09.486451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.486799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.487077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.507047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:09.508458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:09.516713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 928/1835] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:09.521107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.531626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aee917-26c5-442f-85f2-69025a37d7c3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.536361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.536964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.537246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.537522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.538171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:09.538474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.540195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:09.552556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 908/1836] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:09.557098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.566462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a656-3bd0-4724-9711-51343e7ffb84 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.571067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.571660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.571936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.572204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.572832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.574729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:09.672133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 929/1837] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:09 2022] POST /identity/v3/projects => generated 389 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:09.676177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.686207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fbd1-b77b-4a66-8eff-4888bea78064 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.740774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.741296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.741570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.741843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.742466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.744391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:09.841102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 909/1838] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:09 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:09.846012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.856341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05439198-7a16-44de-84c0-0ae8b13930bb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.860981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.861577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.861849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.862117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.862850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.864443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:09.874212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 930/1839] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:09.878160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.888390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6816f-1afc-4a24-b617-021d7ade52e3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.893602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc975eb1-313b-4cf4-a722-bf91c6e3dd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.894200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc975eb1-313b-4cf4-a722-bf91c6e3dd5d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.894482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc975eb1-313b-4cf4-a722-bf91c6e3dd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.894863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc975eb1-313b-4cf4-a722-bf91c6e3dd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.895510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc975eb1-313b-4cf4-a722-bf91c6e3dd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.899859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc975eb1-313b-4cf4-a722-bf91c6e3dd5d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:09.909097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 910/1840] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:09.913205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:09.923107 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f6016-d761-4897-8ef9-eb09f41a4a4d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:09.927606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:09.928264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:09.928538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:09.928803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] PATH_INFO: `/v3/projects/8b9d5ae4f6624198826cd412d0d0ce24/users/ab0373a7c6fd4be7b81e6751a8f6427b/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:09.943500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9d5ae4f6624198826cd412d0d0ce24, user_id=ab0373a7c6fd4be7b81e6751a8f6427b, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:09.946190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.011514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 931/1841] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:09 2022] PUT /identity/v3/projects/8b9d5ae4f6624198826cd412d0d0ce24/users/ab0373a7c6fd4be7b81e6751a8f6427b/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:10.023087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.023371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.023721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.053133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] MFA Rules not processed for user `ab0373a7c6fd4be7b81e6751a8f6427b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:10.054359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:10.069992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:10.127825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 911/1842] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:10.132282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.195114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e111bda-f6dc-4ef7-8e4b-32f48e6f84d5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.199767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.200367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.200646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.200918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.201559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:10.204136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.293928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:10.301611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 932/1843] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:10 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:10.305530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.317992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd53f-8329-4f4c-ab8b-9875b5934122 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.383704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.384541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.384846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.385452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.385775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:10.387766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.481353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 912/1844] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:10 2022] POST /identity/v3/users => generated 301 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:10.492008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.518653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08585a-66f3-452c-9c27-89d88269e30d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.527613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.528075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.528357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.528631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.529280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:10.530947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.542610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 933/1845] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:10.547335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.557624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37e00-89c0-40d6-a792-3c40718a09db admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.563397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.563512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.564195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.564357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] PATH_INFO: `/v3/projects/7810fd2c4c834bfeb561e2ab8cfc6051/users/92f7cff730264128a836bc1450299222/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.580638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7810fd2c4c834bfeb561e2ab8cfc6051, user_id=92f7cff730264128a836bc1450299222, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:10.583306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.675595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 913/1846] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:10 2022] PUT /identity/v3/projects/7810fd2c4c834bfeb561e2ab8cfc6051/users/92f7cff730264128a836bc1450299222/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:10.676817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.688556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bda3-3b7d-4b0d-9f02-b63831eb16f3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.752672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a99e9-9cd1-41b7-9636-1b69766b58b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.753294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a99e9-9cd1-41b7-9636-1b69766b58b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.753572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a99e9-9cd1-41b7-9636-1b69766b58b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.753966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a99e9-9cd1-41b7-9636-1b69766b58b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.754869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a99e9-9cd1-41b7-9636-1b69766b58b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:10.756481 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a99e9-9cd1-41b7-9636-1b69766b58b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.766971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 934/1847] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:10.770907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.790172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df36296-f4bd-437a-a74c-a6d226d6cb18 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.790172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.790172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.790172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.791034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] PATH_INFO: `/v3/projects/7810fd2c4c834bfeb561e2ab8cfc6051/users/92f7cff730264128a836bc1450299222/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.791034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7810fd2c4c834bfeb561e2ab8cfc6051, user_id=92f7cff730264128a836bc1450299222, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:10.793190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:10.804878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 914/1848] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:10 2022] PUT /identity/v3/projects/7810fd2c4c834bfeb561e2ab8cfc6051/users/92f7cff730264128a836bc1450299222/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:10.816620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.816902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.817177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:10.845170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] MFA Rules not processed for user `92f7cff730264128a836bc1450299222`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:10.846396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:10.919710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 935/1849] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:10.930887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:10.932597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:10.990065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb211d8-79bb-459b-938b-bd70fa0ca847 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:10.993813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4cf72-195a-48d9-97a1-4acad14b91da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:10.994790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4cf72-195a-48d9-97a1-4acad14b91da service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:10.995227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4cf72-195a-48d9-97a1-4acad14b91da service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:10.995654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4cf72-195a-48d9-97a1-4acad14b91da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:11.005864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4cf72-195a-48d9-97a1-4acad14b91da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:11.007299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4cf72-195a-48d9-97a1-4acad14b91da service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:11.016525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 915/1850] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:11.151255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:11.214740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3b85a-006d-46f9-a601-4face5c823d0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:11.219423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:11.220057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:11.220330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:11.220603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] PATH_INFO: `/v3/users/ab0373a7c6fd4be7b81e6751a8f6427b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:11.221956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0373a7c6fd4be7b81e6751a8f6427b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:11.224134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:11.237565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:11.408298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0373a7c6fd4be7b81e6751a8f6427b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:11.408959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0373a7c6fd4be7b81e6751a8f6427b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:11.409570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0373a7c6fd4be7b81e6751a8f6427b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:11.416649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0373a7c6fd4be7b81e6751a8f6427b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:11.429836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0373a7c6fd4be7b81e6751a8f6427b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:11.485969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 936/1851] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:11 2022] DELETE /identity/v3/users/ab0373a7c6fd4be7b81e6751a8f6427b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:11.490575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:11.491053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:11.491138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:11.510475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:11.511733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:11.517149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:11.585951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 916/1852] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:11.598028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:11.611533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:11.697398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec80d7-ba87-45e6-b2a2-f662c55e3e28 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:11.702469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:11.703271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:11.703717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:11.704137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:11.715078 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:11.717280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:11.730780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 937/1853] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:12.157300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:12.158844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:12.170925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd8ae-8dc0-4cf4-a858-3c22caff8f55 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:12.175574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:12.176218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:12.179852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:12.179852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] PATH_INFO: `/v3/projects/8b9d5ae4f6624198826cd412d0d0ce24` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:12.179852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9d5ae4f6624198826cd412d0d0ce24)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:12.179852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:12.314842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9d5ae4f6624198826cd412d0d0ce24', 'request_id': 'req-dc0515c6-4144-4a41-911c-16932f1ec56a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.321363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9d5ae4f6624198826cd412d0d0ce24', 'request_id': 'req-dc0515c6-4144-4a41-911c-16932f1ec56a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.327566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9d5ae4f6624198826cd412d0d0ce24', 'request_id': 'req-dc0515c6-4144-4a41-911c-16932f1ec56a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.415791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] The token cache is being invalidate because project 8b9d5ae4f6624198826cd412d0d0ce24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:12.416002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.416615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.418120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 917/1854] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:12 2022] DELETE /identity/v3/projects/8b9d5ae4f6624198826cd412d0d0ce24 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:12.422263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:12.432034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14316f-5888-4b65-b44d-44ff0d879ce6 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:12.487827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:12.488442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:12.488715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:12.488980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] PATH_INFO: `/v3/users/92f7cff730264128a836bc1450299222` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:12.490072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92f7cff730264128a836bc1450299222)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:12.492065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:12.666386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:12.681043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92f7cff730264128a836bc1450299222'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.681699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92f7cff730264128a836bc1450299222'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.682304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92f7cff730264128a836bc1450299222'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.689793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92f7cff730264128a836bc1450299222'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.701601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-84035026-4f48-43c9-af72-99dc138fa4e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92f7cff730264128a836bc1450299222'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:12.794819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 938/1855] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:12 2022] DELETE /identity/v3/users/92f7cff730264128a836bc1450299222 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:13.386003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:13.387486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:13.400980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0515c6-4144-4a41-911c-16932f1ec56a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:13.459069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:13.459692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:13.459966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:13.460235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] PATH_INFO: `/v3/projects/7810fd2c4c834bfeb561e2ab8cfc6051` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:13.461347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7810fd2c4c834bfeb561e2ab8cfc6051)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:13.463267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:13.573830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:13.583151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7810fd2c4c834bfeb561e2ab8cfc6051', 'request_id': 'req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:13.589186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7810fd2c4c834bfeb561e2ab8cfc6051', 'request_id': 'req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:13.595164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7810fd2c4c834bfeb561e2ab8cfc6051', 'request_id': 'req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:13.684754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] The token cache is being invalidate because project 7810fd2c4c834bfeb561e2ab8cfc6051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:13.685043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:13.685669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9ae989fc-3997-47cc-a5a2-eb68b9c1a9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:13.687237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 918/1856] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:13 2022] DELETE /identity/v3/projects/7810fd2c4c834bfeb561e2ab8cfc6051 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:13.692503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:13.693032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:13.694711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:13.714604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:13.715788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:13.774304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 939/1857] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:13.784098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:13.784379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:13.784649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:13.803860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:13.805052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:13.813079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 919/1858] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:13.817005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:13.826440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8a39-597f-4962-84d7-549ff290f493 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:13.831573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:13.832171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:13.832449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:13.832722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:13.833413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:13.833720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:13.835327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:13.847575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 940/1859] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:15:13.851787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:13.861047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be993d2-5f68-45e6-99bd-14fe6eacf4f8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:13.865592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:13.866178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:13.866536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:13.869459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:13.869459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:13.869459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:13.950599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 920/1860] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:13 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:13.954097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:13.964285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af4b7f-face-4add-92df-a5ffc0ae2c2a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:14.022555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:14.023213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.023488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.023758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.024445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:14.026284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:14.106095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 941/1861] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:13 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:14.107038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:14.122022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed10b34-4981-44a0-a8bf-becdc510f8ef admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:14.145527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:14.146150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.146432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.146744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.147402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:14.149063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:14.162621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 921/1862] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:14.164077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:14.174888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1961ee-b1bb-4793-a509-1f5b104bb490 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:14.180050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aeb0e9-b5df-416e-a099-5cb9b4aa854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:14.180676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aeb0e9-b5df-416e-a099-5cb9b4aa854d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.180958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aeb0e9-b5df-416e-a099-5cb9b4aa854d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.181234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aeb0e9-b5df-416e-a099-5cb9b4aa854d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.181896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aeb0e9-b5df-416e-a099-5cb9b4aa854d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:14.183559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aeb0e9-b5df-416e-a099-5cb9b4aa854d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:14.195761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 942/1863] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:14.200405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:14.211770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed8a65-d2cd-41f7-8ae8-e28ea85843ea admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:14.217573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:14.218277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.223725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.223725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] PATH_INFO: `/v3/projects/f8dda6c4980d469e97d63f34ba988ef6/users/fdd4a06dacd541f9949af0c4cd1ddfed/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.240229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dda6c4980d469e97d63f34ba988ef6, user_id=fdd4a06dacd541f9949af0c4cd1ddfed, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:14.244154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:14.376616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 922/1864] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:14 2022] PUT /identity/v3/projects/f8dda6c4980d469e97d63f34ba988ef6/users/fdd4a06dacd541f9949af0c4cd1ddfed/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:14.387626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7b1d9-3d48-495f-8561-18823e92806e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.387912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7b1d9-3d48-495f-8561-18823e92806e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.388219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7b1d9-3d48-495f-8561-18823e92806e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.417715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-83d7b1d9-3d48-495f-8561-18823e92806e None None] MFA Rules not processed for user `fdd4a06dacd541f9949af0c4cd1ddfed`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:14.419763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-83d7b1d9-3d48-495f-8561-18823e92806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:14.424034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:14.499054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 943/1865] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:14.511371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:14.513276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:14.602940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001636e-4ed4-4e28-ae21-3fdc7d79e2e3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:14.606826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:14.608183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.608183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.608183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.618257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c23f-85a8-406c-b382-f225e1053eea None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.618595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c23f-85a8-406c-b382-f225e1053eea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.618844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c23f-85a8-406c-b382-f225e1053eea None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.620407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 944/1866] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:15:14.622088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:14.625273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:14.636332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.636332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.636332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.636732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 923/1867] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:14.657786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:14.659162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:14.745263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 945/1868] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:14.755665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:14.759282 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:14.787890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ce7c-7cc1-48e6-b1ea-d1be1778f2f4 service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:14.794870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:14.795541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:14.802970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:14.802970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:14.805920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:14.807961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:14.819647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 924/1869] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:15.179765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:15.181407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:15.244572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03625f80-e722-4120-bf44-39b85b47f720 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:15.248336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:15.249484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:15.249765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:15.250038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:15.251808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:15.316936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:15.331718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:15.333180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:15.343378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 946/1870] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:16.650363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:16.673913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e1b74-2a55-4fe0-a5b3-4d97f7fa5ac7 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:16.684079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d6351b8d5d4e31ada2cd40c63db8aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b2802bdfbc0465090621102aa9def20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:16.684815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:16.685104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:16.686694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:16.687348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:16.687783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:16.689584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:16.696442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:16.698406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:16.711011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 925/1871] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:16.715868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:16.729188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769a70-7b9d-461f-a1da-d2457fb06aac tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:16.735931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d6351b8d5d4e31ada2cd40c63db8aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b2802bdfbc0465090621102aa9def20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:16.736615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:16.736901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:16.737178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:16.737857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:16.739857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:16.763945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:16.803095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc6a72-9ae6-4e51-a193-81a85a4729a6 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:16.808404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:16.809148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:16.809493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:16.809859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] PATH_INFO: `/v3/users/08c7f501d2be466d8a83033d0e73ef69` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:16.811813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c7f501d2be466d8a83033d0e73ef69)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:16.813955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:16.848432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 926/1872] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:16 2022] POST /identity/v3/projects => generated 354 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:16.853341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:16.976011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0f908-db96-48b3-a1bd-348ea94a5bf7 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:16.982159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d6351b8d5d4e31ada2cd40c63db8aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b2802bdfbc0465090621102aa9def20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:16.983441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:16.983781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:16.984242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:16.984966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:16.985334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:16.987044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:17.006173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 927/1873] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:17.051571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c7f501d2be466d8a83033d0e73ef69'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.052264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c7f501d2be466d8a83033d0e73ef69'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.052869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c7f501d2be466d8a83033d0e73ef69'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.062493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c7f501d2be466d8a83033d0e73ef69'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.089295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:17.090407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424d8b4-5523-4607-bb0a-5a7f5fc7f2d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c7f501d2be466d8a83033d0e73ef69'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.097709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:17.124123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744227be-358e-4b77-85de-fc5f2be1c340 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:17.133231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d6351b8d5d4e31ada2cd40c63db8aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b2802bdfbc0465090621102aa9def20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:17.144205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:17.144205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:17.144205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] PATH_INFO: `/v3/projects/7b311b34abca4a238ee06f7e386a0720` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:17.144205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7b311b34abca4a238ee06f7e386a0720)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:17.145060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:17.194253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 947/1874] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:16 2022] DELETE /identity/v3/users/08c7f501d2be466d8a83033d0e73ef69 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:17.199309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0334-3531-47d2-a97d-e02925874707 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:17.199700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0334-3531-47d2-a97d-e02925874707 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:17.199979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0334-3531-47d2-a97d-e02925874707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:17.220820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-959f0334-3531-47d2-a97d-e02925874707 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:17.222104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-959f0334-3531-47d2-a97d-e02925874707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:17.278207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b311b34abca4a238ee06f7e386a0720', 'request_id': 'req-adcaa569-bb56-4420-aad0-fa32bb597410'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.288358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b311b34abca4a238ee06f7e386a0720', 'request_id': 'req-adcaa569-bb56-4420-aad0-fa32bb597410'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.297299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b311b34abca4a238ee06f7e386a0720', 'request_id': 'req-adcaa569-bb56-4420-aad0-fa32bb597410'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.305157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 948/1875] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:17.315414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f0334-3531-47d2-a97d-e02925874707 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:17.316805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-959f0334-3531-47d2-a97d-e02925874707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:17.327168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:17.379035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f0334-3531-47d2-a97d-e02925874707 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:17.394231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] The token cache is being invalidate because project 7b311b34abca4a238ee06f7e386a0720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:17.394679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.395498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:17.402674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 928/1876] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:17 2022] DELETE /identity/v3/projects/7b311b34abca4a238ee06f7e386a0720 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:17.438724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:17.439340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:17.439618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:17.439999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:17.441788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:17.511711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:17.513286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:17.524945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 949/1877] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:18.009594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:18.011187 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:18.029124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaa569-bb56-4420-aad0-fa32bb597410 tempest-VolumeQuotaClassesTest-1638425526 tempest-VolumeQuotaClassesTest-1638425526-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:18.034455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:18.035205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:18.036026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:18.036026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] PATH_INFO: `/v3/projects/d93d3a908d324b9ca5a05f438146ef0f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:18.037084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93d3a908d324b9ca5a05f438146ef0f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:18.039304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:18.193239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93d3a908d324b9ca5a05f438146ef0f', 'request_id': 'req-d8e66865-af4a-4e63-b277-3b1666eb975f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.205533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93d3a908d324b9ca5a05f438146ef0f', 'request_id': 'req-d8e66865-af4a-4e63-b277-3b1666eb975f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.213345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93d3a908d324b9ca5a05f438146ef0f', 'request_id': 'req-d8e66865-af4a-4e63-b277-3b1666eb975f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.218320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:18.236425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:18.237907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:18.257770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e45ac4-e020-4b9f-9382-fb1dc04119b0 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:18.298461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] The token cache is being invalidate because project d93d3a908d324b9ca5a05f438146ef0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:18.302028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.302028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.303504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 929/1878] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:18 2022] DELETE /identity/v3/projects/d93d3a908d324b9ca5a05f438146ef0f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:18.307448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:18.310046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:18.422779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:18.424028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:18.424433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:18.424808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] PATH_INFO: `/v3/users/20f336c782624850aa0864c99132fbd9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:18.426201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f336c782624850aa0864c99132fbd9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:18.426355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e66865-af4a-4e63-b277-3b1666eb975f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:18.429051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:18.431584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:18.433567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:18.433567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:18.433567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] PATH_INFO: `/v3/users/9e75a0236ded483e826a0520e7ecc315` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:18.434918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e75a0236ded483e826a0520e7ecc315)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:18.437315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:18.451080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:18.662490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f336c782624850aa0864c99132fbd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.662668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e75a0236ded483e826a0520e7ecc315'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.663276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e75a0236ded483e826a0520e7ecc315'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.663386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f336c782624850aa0864c99132fbd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.663955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e75a0236ded483e826a0520e7ecc315'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.664198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f336c782624850aa0864c99132fbd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.672075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e75a0236ded483e826a0520e7ecc315'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.672869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f336c782624850aa0864c99132fbd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.684951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-20475773-6f55-4c85-b25b-edd259d96e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e75a0236ded483e826a0520e7ecc315'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.687265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f336c782624850aa0864c99132fbd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:18.766995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 930/1879] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:18 2022] DELETE /identity/v3/users/9e75a0236ded483e826a0520e7ecc315 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:18.887041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 950/1880] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:18 2022] DELETE /identity/v3/users/20f336c782624850aa0864c99132fbd9 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:18.891561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:18.891869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:18.892166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:18.911973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:18.913253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:18.974067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 931/1881] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:18.986127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:18.991852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:19.081841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553af37-b8f8-43ae-9ddb-4ac5b5c75900 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:19.086788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3fd8d-d3c3-44f0-9d7a-10d93c1489bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:19.087536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3fd8d-d3c3-44f0-9d7a-10d93c1489bf service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.088119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3fd8d-d3c3-44f0-9d7a-10d93c1489bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.088517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3fd8d-d3c3-44f0-9d7a-10d93c1489bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.099608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3fd8d-d3c3-44f0-9d7a-10d93c1489bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:19.101133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3fd8d-d3c3-44f0-9d7a-10d93c1489bf service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:19.112472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 951/1882] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:19.280829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:19.280829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:19.294524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0b6c-e227-4546-93e9-096e8ed6b8ee None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:19.302704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:19.305551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.309002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.309002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] PATH_INFO: `/v3/projects/5d3220aed0de4a52b3fcb0c8b90e8d8d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.309002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3220aed0de4a52b3fcb0c8b90e8d8d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:19.309710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:19.321092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:19.442969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3220aed0de4a52b3fcb0c8b90e8d8d', 'request_id': 'req-3350e128-443e-4030-8128-eb710eb3d75e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:19.457192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3220aed0de4a52b3fcb0c8b90e8d8d', 'request_id': 'req-3350e128-443e-4030-8128-eb710eb3d75e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:19.478091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3220aed0de4a52b3fcb0c8b90e8d8d', 'request_id': 'req-3350e128-443e-4030-8128-eb710eb3d75e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:19.567693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] The token cache is being invalidate because project 5d3220aed0de4a52b3fcb0c8b90e8d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:19.568163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:19.570365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3350e128-443e-4030-8128-eb710eb3d75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:19.572375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 932/1883] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:19 2022] DELETE /identity/v3/projects/5d3220aed0de4a52b3fcb0c8b90e8d8d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:19.578285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.578770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.578922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.651870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:19.653594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:19.734035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 952/1884] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:19.737444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.738067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.738476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.811522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:19.813097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:19.816089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:19.817379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:19.828352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8dbac-e1d1-4f79-8a10-33c19080d0cb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:19.834178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:19.835117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.838620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.838747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 933/1885] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:19.838828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] PATH_INFO: `/v3/users/cb0974ddcc5c463db9a2f4ecd72f7add` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.838828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0974ddcc5c463db9a2f4ecd72f7add)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:19.839097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:19.839749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:19.849352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65b889-48d3-422d-8067-c06cc14615b1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:19.853982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:19.854617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.854902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.855176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.855819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:19.856147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:19.858266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:19.870795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 934/1886] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:15:19.876129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:19.886114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1c21c-e019-4660-99a6-0c63c360f791 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:19.891906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:19.892622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:19.892889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:19.893163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:19.893797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:19.895769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.037333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 935/1887] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:19 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:20.038261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.039915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.060340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:20.075297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0974ddcc5c463db9a2f4ecd72f7add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.076000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0974ddcc5c463db9a2f4ecd72f7add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.076642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0974ddcc5c463db9a2f4ecd72f7add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.083955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0974ddcc5c463db9a2f4ecd72f7add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.096400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0974ddcc5c463db9a2f4ecd72f7add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.104183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8eb6-c1ff-40c4-a958-dc661a77a737 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.111274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.112773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.113297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.113297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] PATH_INFO: `/v3/projects/0c4e175becee44cf9e17422c0ed1b99a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.115810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4e175becee44cf9e17422c0ed1b99a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.117875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.147525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 953/1888] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:19 2022] DELETE /identity/v3/users/cb0974ddcc5c463db9a2f4ecd72f7add => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:20.147525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.147525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.221482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27696a-3263-4e7e-876f-a7afc2b2f6ef admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.226236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e8451-7adb-4b5a-80d7-5ef78ebd82c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.226886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e8451-7adb-4b5a-80d7-5ef78ebd82c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.227168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e8451-7adb-4b5a-80d7-5ef78ebd82c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.227445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e8451-7adb-4b5a-80d7-5ef78ebd82c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.228088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e8451-7adb-4b5a-80d7-5ef78ebd82c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.230006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e8451-7adb-4b5a-80d7-5ef78ebd82c4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.259372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4e175becee44cf9e17422c0ed1b99a', 'request_id': 'req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.265665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4e175becee44cf9e17422c0ed1b99a', 'request_id': 'req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.269530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:20.271696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4e175becee44cf9e17422c0ed1b99a', 'request_id': 'req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.315781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 954/1889] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:20 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:20.317126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.317488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.317769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.337402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:20.338651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.373306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] The token cache is being invalidate because project 0c4e175becee44cf9e17422c0ed1b99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:20.373938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.374420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.376056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 936/1890] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:20 2022] DELETE /identity/v3/projects/0c4e175becee44cf9e17422c0ed1b99a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:20.379605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.380911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.402419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 955/1891] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:20.403495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.405488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.418789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d9d1f-d6a2-4f7f-bffa-fa93f817c8e4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.424192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.424816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.425125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.425447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] PATH_INFO: `/v3/users/f0d6351b8d5d4e31ada2cd40c63db8aa` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.426617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d6351b8d5d4e31ada2cd40c63db8aa)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.428647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.450184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f50cc5-08f0-4f9a-8135-d1967d1dad3b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.455856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.456609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.456915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.457189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.457856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.459596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.470501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 937/1892] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:20.471319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.472964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.530877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17d302-6204-4fe6-886f-b0b43e08445e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.534436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.535068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.535340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.535608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.544622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.546693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.559442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 938/1893] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:20.560128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.569739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cc693-35bc-42b6-8a17-70a4ff3f8571 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.575118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.575930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.576410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.576688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.577328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.579003 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.589526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 939/1894] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:20.594527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.605061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194872e4-bb84-48d1-a81a-3e58988fd094 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:20.610672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477a87-aebd-46a5-b6f1-5e9e02f17ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:20.611495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477a87-aebd-46a5-b6f1-5e9e02f17ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.611775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477a87-aebd-46a5-b6f1-5e9e02f17ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.612043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477a87-aebd-46a5-b6f1-5e9e02f17ba8 admin admin] PATH_INFO: `/v3/projects/38dda6facad54c72b3df641bbe6fb052/users/c037c75ba6a54f229c4b97e1f910c8ac/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.631043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477a87-aebd-46a5-b6f1-5e9e02f17ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38dda6facad54c72b3df641bbe6fb052, user_id=c037c75ba6a54f229c4b97e1f910c8ac, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:20.633917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477a87-aebd-46a5-b6f1-5e9e02f17ba8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:20.699779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d6351b8d5d4e31ada2cd40c63db8aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.700619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d6351b8d5d4e31ada2cd40c63db8aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.701463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d6351b8d5d4e31ada2cd40c63db8aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.710260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d6351b8d5d4e31ada2cd40c63db8aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.713676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 940/1895] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:20 2022] PUT /identity/v3/projects/38dda6facad54c72b3df641bbe6fb052/users/c037c75ba6a54f229c4b97e1f910c8ac/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:20.730589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:20.730765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:20.734075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:20.738282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d6351b8d5d4e31ada2cd40c63db8aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:20.763140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] MFA Rules not processed for user `c037c75ba6a54f229c4b97e1f910c8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:20.764397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.857294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 956/1896] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:20 2022] DELETE /identity/v3/users/f0d6351b8d5d4e31ada2cd40c63db8aa => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:20.873920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:20.939707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 941/1897] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:20.944588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:20.946163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:20.961881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:21.061212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d14c7-13f4-4889-80ac-e639a070ba9f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.066425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.067222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.067508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.067839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.068421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.070308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.145060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.146679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:21.176963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 957/1898] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:20 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:21.183218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.242648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b5078-1d72-4dc9-87db-0e9c98881157 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.298138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae157490-ec93-48cc-b14c-5e7c79fbc0a2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.304260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.305721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.306153 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.306620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.307887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.309349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.320532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.321805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.322318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.322902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] PATH_INFO: `/v3/projects/a8aeafd059ed4c9e9c5fcaa4e5323742` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.325999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8aeafd059ed4c9e9c5fcaa4e5323742)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.329910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.413054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 958/1899] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:21 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:21.417231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.429655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed610db-50e4-41c2-aa81-b21518cba57f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.435075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.436567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.436991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.437635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.438511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.441175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.460296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 959/1900] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:21.461190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.467324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8aeafd059ed4c9e9c5fcaa4e5323742', 'request_id': 'req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.474174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7097d-b6c9-4389-b680-a9d6e55f9780 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.483380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8aeafd059ed4c9e9c5fcaa4e5323742', 'request_id': 'req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.489768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8aeafd059ed4c9e9c5fcaa4e5323742', 'request_id': 'req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.547834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.548672 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.548901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.549245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] PATH_INFO: `/v3/projects/2fb6545cd0f840b5bdecb56172fdc5ea/users/c646a190893549df87a945b3954b0e8e/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.566745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb6545cd0f840b5bdecb56172fdc5ea, user_id=c646a190893549df87a945b3954b0e8e, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.569694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.618893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:21.624418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] The token cache is being invalidate because project a8aeafd059ed4c9e9c5fcaa4e5323742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:21.624719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.625394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.627183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 942/1901] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:21 2022] DELETE /identity/v3/projects/a8aeafd059ed4c9e9c5fcaa4e5323742 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:21.627939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.629414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:21.643326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f1ec-13f4-4cac-a72b-ca2de34014b2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.648217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.648928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.649208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.649513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] PATH_INFO: `/v3/projects/0b2802bdfbc0465090621102aa9def20` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.650949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2802bdfbc0465090621102aa9def20)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.653170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.711137 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 960/1902] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:21 2022] PUT /identity/v3/projects/2fb6545cd0f840b5bdecb56172fdc5ea/users/c646a190893549df87a945b3954b0e8e/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:21.713864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.715569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:21.795572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:21.795799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9b21a-636a-4fde-8bf8-fc65027926c7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:21.795799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:21.795799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:21.795799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:21.796155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] PATH_INFO: `/v3/users/e727fbd24c5f48f398cafdd0eb585863` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:21.796155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e727fbd24c5f48f398cafdd0eb585863)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:21.796155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:21.882018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2802bdfbc0465090621102aa9def20', 'request_id': 'req-08db2626-d29c-47ea-aca4-3ee4e0e681f6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.888252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2802bdfbc0465090621102aa9def20', 'request_id': 'req-08db2626-d29c-47ea-aca4-3ee4e0e681f6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.896372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2802bdfbc0465090621102aa9def20', 'request_id': 'req-08db2626-d29c-47ea-aca4-3ee4e0e681f6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.988124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] The token cache is being invalidate because project 0b2802bdfbc0465090621102aa9def20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:21.988416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.989041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:21.990566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 943/1903] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:21 2022] DELETE /identity/v3/projects/0b2802bdfbc0465090621102aa9def20 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:21.991374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:21.993179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:22.006317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e727fbd24c5f48f398cafdd0eb585863'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:22.008745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e727fbd24c5f48f398cafdd0eb585863'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:22.008745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e727fbd24c5f48f398cafdd0eb585863'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:22.017723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e727fbd24c5f48f398cafdd0eb585863'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:22.031304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c365172-e1e1-486c-b985-953141d49b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e727fbd24c5f48f398cafdd0eb585863'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:22.065400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db2626-d29c-47ea-aca4-3ee4e0e681f6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.070220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80ec6e-cae7-44ed-a236-7e4d370fa2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.070865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80ec6e-cae7-44ed-a236-7e4d370fa2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.071147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80ec6e-cae7-44ed-a236-7e4d370fa2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.071568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80ec6e-cae7-44ed-a236-7e4d370fa2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.072232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80ec6e-cae7-44ed-a236-7e4d370fa2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.073846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80ec6e-cae7-44ed-a236-7e4d370fa2f7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.084085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 944/1904] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:22.089073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.089073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.089073 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.104450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:22.105656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:22.125778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 945/1905] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:22.125778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.125778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:22.139230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dea9b-5d16-43bc-ab08-1e04900749e7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.143534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a787a-2876-400e-b7f9-58c317259f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.146538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a787a-2876-400e-b7f9-58c317259f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.146538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a787a-2876-400e-b7f9-58c317259f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.146538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a787a-2876-400e-b7f9-58c317259f20 admin admin] PATH_INFO: `/v3/projects/2fb6545cd0f840b5bdecb56172fdc5ea/users/c646a190893549df87a945b3954b0e8e/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.146921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a787a-2876-400e-b7f9-58c317259f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb6545cd0f840b5bdecb56172fdc5ea, user_id=c646a190893549df87a945b3954b0e8e, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.149474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a787a-2876-400e-b7f9-58c317259f20 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.155125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 961/1906] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:21 2022] DELETE /identity/v3/users/e727fbd24c5f48f398cafdd0eb585863 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:22.155125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.155125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.155125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.192414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 946/1907] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:22 2022] PUT /identity/v3/projects/2fb6545cd0f840b5bdecb56172fdc5ea/users/c646a190893549df87a945b3954b0e8e/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:22.192581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:22.192581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:22.202598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.203398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.203684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.241212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] MFA Rules not processed for user `c646a190893549df87a945b3954b0e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:22.242438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:22.250711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:22.311596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 962/1908] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:22.316096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.338843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e61dc2-d564-463d-aa8c-cdaa19f09fe0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.338843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.338843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.338843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.339216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.339216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:22.339216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.339460 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 947/1909] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:22.339547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.347805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.350529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:22.353525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 963/1910] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:22.362990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:22.368142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.381557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dea952-a9ed-4781-b9ed-5fee069acf05 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.389088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.389705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.389984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.390258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.390952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.394093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.462835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdfb75-a0d8-4331-8707-e8fb41bde027 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.467536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.468170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.468447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.468720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.489197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.490682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.515673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 964/1912] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:22 2022] POST /identity/v3/projects => generated 399 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:22.518336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 948/1911] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:22.519802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.620433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44457d8-23a0-4efd-9124-924e85efadaf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.626605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.627464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.628044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.628821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.629733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.647632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.722212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 965/1913] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:22 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:22.726305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.738325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71066e8-6d12-4e98-b140-4d59180fef88 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.743498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.748910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.748910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.748910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.748910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.748910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.757917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 949/1914] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:22.762174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.777464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d684b3-4821-4dc6-b4f0-60e8e390ef5d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.782933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.783666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.784042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.784379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.785114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.786800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.801359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 966/1915] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:22.808847 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.825335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26f8f-1fff-4d22-b95f-7da96fe2a758 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.830303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302de22-df96-4502-9568-516c5b6bb577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:22.832764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302de22-df96-4502-9568-516c5b6bb577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.832764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302de22-df96-4502-9568-516c5b6bb577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.832764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302de22-df96-4502-9568-516c5b6bb577 admin admin] PATH_INFO: `/v3/projects/5d8dd687aea942199df2a16ddc012be0/users/14e7255dd08a481aaa9afec8e8df74a0/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:22.847275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302de22-df96-4502-9568-516c5b6bb577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8dd687aea942199df2a16ddc012be0, user_id=14e7255dd08a481aaa9afec8e8df74a0, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:22.849921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302de22-df96-4502-9568-516c5b6bb577 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:22.859049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:22.968404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe2e3b-1265-47c9-bf7d-a7a6e9eb2044 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:22.969093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 950/1916] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:22 2022] PUT /identity/v3/projects/5d8dd687aea942199df2a16ddc012be0/users/14e7255dd08a481aaa9afec8e8df74a0/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:22.980171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:22.981948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:22.983348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:22.983850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.032425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] MFA Rules not processed for user `14e7255dd08a481aaa9afec8e8df74a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:23.035303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:23.039362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.040074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.040409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.040756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.042800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:23.129520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 951/1917] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:23.130492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.133592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:23.157547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.159416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.173317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 967/1918] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:23.174322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.227389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8550a1-d985-495b-8c0c-62406ccfca1d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:23.261940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.265288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.266804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.268432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] PATH_INFO: `/v3/projects/3efa775b67604802b6a64961d8a77dab` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.271272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efa775b67604802b6a64961d8a77dab)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.273946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.276133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:23.292380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2722f43-2e25-4e94-9761-86a8d2fd70bd service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:23.297548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.298312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.298699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.298913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.300313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.302662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.409942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 968/1919] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:23 2022] POST /identity/v3/projects => generated 401 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:23.418973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.440057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efa775b67604802b6a64961d8a77dab', 'request_id': 'req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.447165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efa775b67604802b6a64961d8a77dab', 'request_id': 'req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.454967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efa775b67604802b6a64961d8a77dab', 'request_id': 'req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.505537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3564d50-0c58-4689-9bc0-4faedfbd760d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:23.570675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] The token cache is being invalidate because project 3efa775b67604802b6a64961d8a77dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:23.572830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.572830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.573808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 952/1920] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:23 2022] DELETE /identity/v3/projects/3efa775b67604802b6a64961d8a77dab => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:23.578119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.579494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:23.585872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.586792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.587079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.587356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.588031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.594591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.641806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:23.657402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf4ba-983a-48ce-a55b-98c1cd2e03d4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:23.662402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.663082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.663677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.663677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] PATH_INFO: `/v3/users/1ab7accda2bf42439f155cea70debdc8` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.665260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab7accda2bf42439f155cea70debdc8)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.667267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.708873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 969/1921] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:23 2022] POST /identity/v3/users => generated 307 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:23.711946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.718728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:23.735212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bf3dc-3985-4327-a62b-e18b1c126883 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:23.741210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.742165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.742681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.743138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.744092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.746400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.761428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 970/1922] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:23.765936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.777299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f8490-ce40-4d14-8d6b-d9b70476ca56 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:23.784386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:23.785057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:23.785338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:23.785617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] PATH_INFO: `/v3/projects/d59d1454779e43e9b0f61c74d0047da4/users/5bae62f1d81f4110a761c1dc1ca406e5/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:23.823239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59d1454779e43e9b0f61c74d0047da4, user_id=5bae62f1d81f4110a761c1dc1ca406e5, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:23.823239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:23.883464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab7accda2bf42439f155cea70debdc8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.884188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab7accda2bf42439f155cea70debdc8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.884815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab7accda2bf42439f155cea70debdc8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.893352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab7accda2bf42439f155cea70debdc8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.908656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab7accda2bf42439f155cea70debdc8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:23.944708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 971/1923] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:23 2022] PUT /identity/v3/projects/d59d1454779e43e9b0f61c74d0047da4/users/5bae62f1d81f4110a761c1dc1ca406e5/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:23.949818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:23.951644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:24.024308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 953/1924] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:23 2022] DELETE /identity/v3/users/1ab7accda2bf42439f155cea70debdc8 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:24.064505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d8a72-deef-4b0c-af4c-6f5d86cefced admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:24.157172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ee29a-01d8-4695-8340-90079b694213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:24.158888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ee29a-01d8-4695-8340-90079b694213 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:24.160005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ee29a-01d8-4695-8340-90079b694213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:24.160457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ee29a-01d8-4695-8340-90079b694213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:24.161587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ee29a-01d8-4695-8340-90079b694213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:24.163782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ee29a-01d8-4695-8340-90079b694213 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:24.198069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 972/1925] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:24.203588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:24.231940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d1db2-1bbb-4e36-9712-65d6bde457b2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:24.238355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:24.240080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:24.240080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:24.240810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] PATH_INFO: `/v3/projects/d59d1454779e43e9b0f61c74d0047da4/users/5bae62f1d81f4110a761c1dc1ca406e5/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:24.242474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59d1454779e43e9b0f61c74d0047da4, user_id=5bae62f1d81f4110a761c1dc1ca406e5, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:24.245638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:24.249924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:24.260954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 954/1926] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:24 2022] PUT /identity/v3/projects/d59d1454779e43e9b0f61c74d0047da4/users/5bae62f1d81f4110a761c1dc1ca406e5/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:24.277966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:24.278697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:24.278839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:24.318015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] MFA Rules not processed for user `5bae62f1d81f4110a761c1dc1ca406e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:24.319419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:24.408075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 973/1927] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:24.419119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:24.421966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:24.513881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ed051-6f76-4580-ab16-4db47cd82182 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:24.522866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:24.525027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b84cc-1a3f-4fef-960c-b659c32b4705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:24.525667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b84cc-1a3f-4fef-960c-b659c32b4705 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:24.525948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b84cc-1a3f-4fef-960c-b659c32b4705 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:24.526223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b84cc-1a3f-4fef-960c-b659c32b4705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:24.536855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b84cc-1a3f-4fef-960c-b659c32b4705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:24.538372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b84cc-1a3f-4fef-960c-b659c32b4705 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:24.550197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 955/1928] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:25.011382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:25.013149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:25.142038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e75f04-7ff3-4e89-9f07-1c71e274e8c9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:25.153976 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:25.154857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.155109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.155378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] PATH_INFO: `/v3/projects/6e4662b399604da29439da207cc5de68` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.158127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4662b399604da29439da207cc5de68)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:25.160240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:25.272678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4662b399604da29439da207cc5de68', 'request_id': 'req-641850dd-7ec0-4ea8-afe5-ad53c32cadca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:25.344226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4662b399604da29439da207cc5de68', 'request_id': 'req-641850dd-7ec0-4ea8-afe5-ad53c32cadca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:25.352040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4662b399604da29439da207cc5de68', 'request_id': 'req-641850dd-7ec0-4ea8-afe5-ad53c32cadca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:25.364294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] The token cache is being invalidate because project 6e4662b399604da29439da207cc5de68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:25.364596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:25.365281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-641850dd-7ec0-4ea8-afe5-ad53c32cadca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:25.367128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 974/1929] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:25 2022] DELETE /identity/v3/projects/6e4662b399604da29439da207cc5de68 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:25.372923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.373401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.373518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.396583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:25.397978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:25.472949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 956/1930] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:25.485325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.485607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.485881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.501698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:25.503224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:25.532734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:25.534051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:25.543824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 975/1931] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:25.548442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:25.558435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f5370-540e-4055-992a-57b580e8c53c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:25.571189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:25.571570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.571842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.572167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.573519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:25.573843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:25.577258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:25.582457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:25.595200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a8a06-260e-47e6-83e1-30216114e926 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:25.599996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:25.606821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 976/1932] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:25.606942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.606942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.606942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.606942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:25.607357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:25.623964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1dfb3-1668-42af-95fb-48cfd89151e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:25.632518 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:25.633068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.633439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.633608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.634652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:25.636822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:25.718430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:25.723052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:25.724568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:25.735294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 957/1933] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:25.793458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 977/1934] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:25 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:25.797626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:25.813495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b258-01f4-4f4f-8154-51c2058b1df9 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:25.892960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:25.893764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:25.894565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:25.895229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:25.896997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:25.905831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.110447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 958/1935] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:25 2022] POST /identity/v3/users => generated 290 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:26.115489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.128931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0856-e1b4-43cf-828a-1b191420ed61 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.134100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.137600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.137600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.137600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.137600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.137928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.148683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 978/1936] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:26.152891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.165413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe6fb4-00a5-46ae-ba5a-c708b9f5627f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.178919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346285a9-5bd6-47f3-b2e4-0546dbdb24fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.179546 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346285a9-5bd6-47f3-b2e4-0546dbdb24fe admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.179821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346285a9-5bd6-47f3-b2e4-0546dbdb24fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.180094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346285a9-5bd6-47f3-b2e4-0546dbdb24fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.180749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346285a9-5bd6-47f3-b2e4-0546dbdb24fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.182354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346285a9-5bd6-47f3-b2e4-0546dbdb24fe admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.192910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 959/1937] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:26.197297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.207738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cd116-3176-48a8-b037-7a8cea4ce114 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.212495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.213163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.213445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.213721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] PATH_INFO: `/v3/projects/59028111550e4886a13f83ee2d2a7fa1/users/7dce583b470a409f96462e9e85924fbb/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.229797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=59028111550e4886a13f83ee2d2a7fa1, user_id=7dce583b470a409f96462e9e85924fbb, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.233083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.322173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 979/1938] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:26 2022] PUT /identity/v3/projects/59028111550e4886a13f83ee2d2a7fa1/users/7dce583b470a409f96462e9e85924fbb/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:26.334397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.334718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.335048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.365734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] MFA Rules not processed for user `7dce583b470a409f96462e9e85924fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:26.367061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:26.437441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 960/1939] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:26.442604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.516446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c227af-36d4-4b9d-96af-a03cd72665cb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.521232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.521903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.522247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.522605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.523335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.525306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.642847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 980/1940] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:26 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:26.647060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.658186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cb378-92be-4ab6-a002-74e6ede7f3b9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.731007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.731627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.731908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.732216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.732848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.734859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.743795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:26.844390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 961/1941] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:26 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:26.848466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.858920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd59a2-2c88-4c7a-ad67-62d6d7a80c85 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.863471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.864111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.864385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.865162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.865344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.866967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.877947 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 981/1942] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:26.881706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.892949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0866c8-519a-468f-9bee-a7110877e8e2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.897605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ae441-f23c-4e83-afa2-1a3803806fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.898193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ae441-f23c-4e83-afa2-1a3803806fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.898465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ae441-f23c-4e83-afa2-1a3803806fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.898769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ae441-f23c-4e83-afa2-1a3803806fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.899406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ae441-f23c-4e83-afa2-1a3803806fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.901108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ae441-f23c-4e83-afa2-1a3803806fcf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:26.911958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 962/1943] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:26.916148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:26.926297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63214af3-0399-4db1-bd4a-b2daec79c603 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:26.930857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:26.931506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:26.931781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:26.932054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] PATH_INFO: `/v3/projects/65b721abd6554b389a3c309306dcf1a0/users/05758b1e49944b7a97d4e802a8dbcc17/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:26.948937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b721abd6554b389a3c309306dcf1a0, user_id=05758b1e49944b7a97d4e802a8dbcc17, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:26.952510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.029244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 982/1944] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:26 2022] PUT /identity/v3/projects/65b721abd6554b389a3c309306dcf1a0/users/05758b1e49944b7a97d4e802a8dbcc17/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:27.039486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.039768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.040043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.068487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] MFA Rules not processed for user `05758b1e49944b7a97d4e802a8dbcc17`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:27.069695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:27.134500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 963/1945] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:27.139026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.188860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:27.201886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5a09a-1e4f-49e9-b9a8-052a1fda94ef admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.206751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.207363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.207642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.207912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.208546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.210451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.298047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 983/1946] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:27 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:27.301992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.313384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8456-e4e7-4d14-a261-249a464ef9bc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.370664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.371902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.372178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.372453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.373723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.375679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.446903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 964/1947] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:27 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:27.451391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.461589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a406c8-5fe3-4735-b9ff-33444bc5fcc3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.466774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.467475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.468663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.468663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.468663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.470238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.480312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 984/1948] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:27.485272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.495738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777133a7-8baf-4170-beee-6ba184269961 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.502619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.503546 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.503905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.504254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] PATH_INFO: `/v3/projects/74a6c7bfb1e7454caa9b9fc588b4eae2/users/f52d5e7526cf485299d3858afb4decf3/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.520057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a6c7bfb1e7454caa9b9fc588b4eae2, user_id=f52d5e7526cf485299d3858afb4decf3, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.522783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.616668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 965/1949] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:27 2022] PUT /identity/v3/projects/74a6c7bfb1e7454caa9b9fc588b4eae2/users/f52d5e7526cf485299d3858afb4decf3/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:27.620562 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.630885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05cb55-7614-49a3-bee4-a0078298e3cf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.690313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225761d-7178-47f3-8e09-b456090a7b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.691083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225761d-7178-47f3-8e09-b456090a7b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.691368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225761d-7178-47f3-8e09-b456090a7b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.691640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225761d-7178-47f3-8e09-b456090a7b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.692388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225761d-7178-47f3-8e09-b456090a7b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.694130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225761d-7178-47f3-8e09-b456090a7b47 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.708055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 985/1950] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:27.708983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.720319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9706e52-8528-40c9-b79d-35934111eb0e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.725201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.727382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.727382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.727382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] PATH_INFO: `/v3/projects/74a6c7bfb1e7454caa9b9fc588b4eae2/users/f52d5e7526cf485299d3858afb4decf3/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.729335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a6c7bfb1e7454caa9b9fc588b4eae2, user_id=f52d5e7526cf485299d3858afb4decf3, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.737352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.759389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 966/1951] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:27 2022] PUT /identity/v3/projects/74a6c7bfb1e7454caa9b9fc588b4eae2/users/f52d5e7526cf485299d3858afb4decf3/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:27.770966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.771361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.771772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.813158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] MFA Rules not processed for user `f52d5e7526cf485299d3858afb4decf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:27.813158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:27.884300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 986/1952] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:27.896592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:27.928325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:27.961223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c111dcc-ec20-4f8e-b4f9-e439a750c5cd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:27.965634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:27.966362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:27.966688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:27.966959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:27.977205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:27.980370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:27.988805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 967/1953] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:28.093876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:28.095274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:28.107424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf5f6c-42c0-4728-9fc3-8416f31a14fb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:28.110999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bc1b0-4054-4c8c-916f-77d752812eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:28.111633 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc1b0-4054-4c8c-916f-77d752812eba service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:28.111913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc1b0-4054-4c8c-916f-77d752812eba service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:28.112186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc1b0-4054-4c8c-916f-77d752812eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:28.158936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:28.175767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bc1b0-4054-4c8c-916f-77d752812eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:28.177198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bc1b0-4054-4c8c-916f-77d752812eba service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:28.186057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 987/1954] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:32.273028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:32.274427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:32.339868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4650f84-0f45-4f9a-8fe0-9204a5e365ad service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:32.344949 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:32.345562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:32.345840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:32.346110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] PATH_INFO: `/v3/users/c037c75ba6a54f229c4b97e1f910c8ac` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:32.347234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c037c75ba6a54f229c4b97e1f910c8ac)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:32.349142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:32.522539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c037c75ba6a54f229c4b97e1f910c8ac'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.526944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c037c75ba6a54f229c4b97e1f910c8ac'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.526944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c037c75ba6a54f229c4b97e1f910c8ac'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.532160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c037c75ba6a54f229c4b97e1f910c8ac'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.544669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c037c75ba6a54f229c4b97e1f910c8ac'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.661220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 968/1955] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:32 2022] DELETE /identity/v3/users/c037c75ba6a54f229c4b97e1f910c8ac => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:32.666047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:32.666602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:32.666698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:32.687007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:32.688260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:32.735525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:32.737302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:32.754248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 988/1956] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:32.766911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:32.768477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:32.789336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:32.807864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70636f50-ecef-44bd-8408-d83a347a9c9b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:32.813159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:32.813795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:32.814072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:32.814344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] PATH_INFO: `/v3/users/7dce583b470a409f96462e9e85924fbb` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:32.815640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dce583b470a409f96462e9e85924fbb)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:32.817556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:32.830324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac22c64-6634-4c1a-a0f2-db24385b26a3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:32.835510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0f6d9-cb26-48a8-9696-9d98b1584878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:32.836125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0f6d9-cb26-48a8-9696-9d98b1584878 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:32.836435 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0f6d9-cb26-48a8-9696-9d98b1584878 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:32.836837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0f6d9-cb26-48a8-9696-9d98b1584878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:32.847524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0f6d9-cb26-48a8-9696-9d98b1584878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:32.849277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0f6d9-cb26-48a8-9696-9d98b1584878 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:32.860510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 989/1957] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:32.992179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dce583b470a409f96462e9e85924fbb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.992845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dce583b470a409f96462e9e85924fbb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:32.993465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dce583b470a409f96462e9e85924fbb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.001289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dce583b470a409f96462e9e85924fbb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.020505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dce583b470a409f96462e9e85924fbb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.184557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 969/1958] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:32 2022] DELETE /identity/v3/users/7dce583b470a409f96462e9e85924fbb => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:33.187336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:33.187336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:33.187336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:33.228969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:33.243301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:33.343726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 990/1959] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:33.353628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:33.355257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:33.424202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37755e8-29a0-4f24-a0ab-cfa9b726191b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:33.428778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:33.429723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:33.430198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:33.430715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:33.443284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:33.444010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:33.444821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:33.445690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:33.461568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b87df-ac8d-45c7-8397-0376b9f998b4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:33.463589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 970/1960] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:33.467453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52130750-bb13-404f-856c-4e016883469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:33.469587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52130750-bb13-404f-856c-4e016883469a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:33.469587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52130750-bb13-404f-856c-4e016883469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:33.469587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52130750-bb13-404f-856c-4e016883469a admin admin] PATH_INFO: `/v3/users/14e7255dd08a481aaa9afec8e8df74a0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:33.470256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52130750-bb13-404f-856c-4e016883469a admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e7255dd08a481aaa9afec8e8df74a0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:33.472313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52130750-bb13-404f-856c-4e016883469a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:33.500072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:33.686617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:33.688597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:33.710165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb44548-f19d-4c54-b0e4-416c6b98c949 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:33.722919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:33.723565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:33.724201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:33.725120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] PATH_INFO: `/v3/projects/38dda6facad54c72b3df641bbe6fb052` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:33.726543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38dda6facad54c72b3df641bbe6fb052)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:33.729703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:33.769224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52130750-bb13-404f-856c-4e016883469a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e7255dd08a481aaa9afec8e8df74a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.770630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52130750-bb13-404f-856c-4e016883469a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e7255dd08a481aaa9afec8e8df74a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.771030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52130750-bb13-404f-856c-4e016883469a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e7255dd08a481aaa9afec8e8df74a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.785501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52130750-bb13-404f-856c-4e016883469a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e7255dd08a481aaa9afec8e8df74a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.799759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52130750-bb13-404f-856c-4e016883469a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e7255dd08a481aaa9afec8e8df74a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.876933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38dda6facad54c72b3df641bbe6fb052', 'request_id': 'req-f179ff1c-0124-4491-9cfc-d600589db890'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.883325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38dda6facad54c72b3df641bbe6fb052', 'request_id': 'req-f179ff1c-0124-4491-9cfc-d600589db890'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.888979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:33.894457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38dda6facad54c72b3df641bbe6fb052', 'request_id': 'req-f179ff1c-0124-4491-9cfc-d600589db890'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:33.894787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 991/1961] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:33 2022] DELETE /identity/v3/users/14e7255dd08a481aaa9afec8e8df74a0 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:33.897348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:33.897633 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:33.897905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:33.917552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:33.923291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:33.977293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 992/1962] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:33.986938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:34.001628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] The token cache is being invalidate because project 38dda6facad54c72b3df641bbe6fb052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:34.001628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.001958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:34.002044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.002044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 971/1963] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:33 2022] DELETE /identity/v3/projects/38dda6facad54c72b3df641bbe6fb052 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:34.002044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:34.003698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:34.066648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c10257-4cbd-467d-b56c-5c3d0e968038 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:34.070361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:34.070989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:34.071316 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:34.071534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:34.073465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:34.076074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179ff1c-0124-4491-9cfc-d600589db890 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:34.081030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:34.084338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:34.084338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:34.084338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] PATH_INFO: `/v3/projects/59028111550e4886a13f83ee2d2a7fa1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:34.084338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59028111550e4886a13f83ee2d2a7fa1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:34.085410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:34.086670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:34.088230 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:34.099352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 993/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:34.100245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:34.101560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:34.114222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6c79f-5993-40a2-a31b-ec5c67ea5321 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:34.119080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:34.119757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:34.120024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:34.120296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] PATH_INFO: `/v3/users/c646a190893549df87a945b3954b0e8e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:34.121447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c646a190893549df87a945b3954b0e8e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:34.123573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:34.202401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59028111550e4886a13f83ee2d2a7fa1', 'request_id': 'req-55337929-50c5-46df-81d0-690df4424cb9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.215912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59028111550e4886a13f83ee2d2a7fa1', 'request_id': 'req-55337929-50c5-46df-81d0-690df4424cb9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.222843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59028111550e4886a13f83ee2d2a7fa1', 'request_id': 'req-55337929-50c5-46df-81d0-690df4424cb9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.311607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:34.313329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] The token cache is being invalidate because project 59028111550e4886a13f83ee2d2a7fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:34.313626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.314427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.316294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 972/1965] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:34 2022] DELETE /identity/v3/projects/59028111550e4886a13f83ee2d2a7fa1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:34.319989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:34.321498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:34.323846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c646a190893549df87a945b3954b0e8e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.324542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c646a190893549df87a945b3954b0e8e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.326068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c646a190893549df87a945b3954b0e8e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.336023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c646a190893549df87a945b3954b0e8e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.369967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c646a190893549df87a945b3954b0e8e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.412709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337929-50c5-46df-81d0-690df4424cb9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:34.417368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:34.418011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:34.419203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:34.419203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] PATH_INFO: `/v3/users/05758b1e49944b7a97d4e802a8dbcc17` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:34.420065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05758b1e49944b7a97d4e802a8dbcc17)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:34.422828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:34.469775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 994/1966] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:34 2022] DELETE /identity/v3/users/c646a190893549df87a945b3954b0e8e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:34.604272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:34.639411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:34.640808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:34.643533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05758b1e49944b7a97d4e802a8dbcc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.644574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05758b1e49944b7a97d4e802a8dbcc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.645036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05758b1e49944b7a97d4e802a8dbcc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.674242 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05758b1e49944b7a97d4e802a8dbcc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.704731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05758b1e49944b7a97d4e802a8dbcc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.735606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dff8c1-2bad-4bbd-8e6f-e881d4a17b2d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:34.740523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:34.741163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:34.741437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:34.741738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] PATH_INFO: `/v3/projects/5d8dd687aea942199df2a16ddc012be0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:34.742889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8dd687aea942199df2a16ddc012be0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:34.744759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:34.817467 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 973/1967] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:34 2022] DELETE /identity/v3/users/05758b1e49944b7a97d4e802a8dbcc17 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:34.879000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8dd687aea942199df2a16ddc012be0', 'request_id': 'req-2f267a00-4be2-4434-8353-d3df3ff8eaaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.963926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8dd687aea942199df2a16ddc012be0', 'request_id': 'req-2f267a00-4be2-4434-8353-d3df3ff8eaaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.970962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8dd687aea942199df2a16ddc012be0', 'request_id': 'req-2f267a00-4be2-4434-8353-d3df3ff8eaaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.981556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] The token cache is being invalidate because project 5d8dd687aea942199df2a16ddc012be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:34.981875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.982671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:34.984646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 995/1968] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:34 2022] DELETE /identity/v3/projects/5d8dd687aea942199df2a16ddc012be0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:34.988379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:34.990014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.056265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:35.062140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.065366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e37aa2-f0a2-4cff-8df2-55598554a310 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:35.107227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:35.127193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f267a00-4be2-4434-8353-d3df3ff8eaaf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:35.131025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:35.132369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:35.133158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.133272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.133471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.133571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.133718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] PATH_INFO: `/v3/users/6c77a5189ed14e55b43a5f6eaf860db5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.134258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] PATH_INFO: `/v3/users/5bae62f1d81f4110a761c1dc1ca406e5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.134909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c77a5189ed14e55b43a5f6eaf860db5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:35.135399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bae62f1d81f4110a761c1dc1ca406e5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:35.137067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:35.139182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:35.319900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c77a5189ed14e55b43a5f6eaf860db5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.320288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bae62f1d81f4110a761c1dc1ca406e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.320607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c77a5189ed14e55b43a5f6eaf860db5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.321723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c77a5189ed14e55b43a5f6eaf860db5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.322131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bae62f1d81f4110a761c1dc1ca406e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.322828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bae62f1d81f4110a761c1dc1ca406e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.324453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:35.330396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c77a5189ed14e55b43a5f6eaf860db5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.330724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bae62f1d81f4110a761c1dc1ca406e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.345996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c77a5189ed14e55b43a5f6eaf860db5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.347394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bae62f1d81f4110a761c1dc1ca406e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.424296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 996/1969] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:35 2022] DELETE /identity/v3/users/6c77a5189ed14e55b43a5f6eaf860db5 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:35.424977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:35.426313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.484729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 974/1970] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:34 2022] DELETE /identity/v3/users/5bae62f1d81f4110a761c1dc1ca406e5 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:35.485510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:35.486957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.499927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccea3f-b944-4262-800a-db4989c7c84d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:35.563642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:35.564460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.564808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.565177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] PATH_INFO: `/v3/projects/2fb6545cd0f840b5bdecb56172fdc5ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.566685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb6545cd0f840b5bdecb56172fdc5ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:35.568895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:35.569842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86373f2-b9a3-449b-a68a-9f4c74a906b6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:35.575322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:35.575970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.576287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.576570 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] PATH_INFO: `/v3/projects/65b721abd6554b389a3c309306dcf1a0` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.579332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b721abd6554b389a3c309306dcf1a0)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:35.581490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:35.725654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb6545cd0f840b5bdecb56172fdc5ea', 'request_id': 'req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.734765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b721abd6554b389a3c309306dcf1a0', 'request_id': 'req-17084de9-1a17-45d3-9def-23ce8e6f70fa'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.741825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b721abd6554b389a3c309306dcf1a0', 'request_id': 'req-17084de9-1a17-45d3-9def-23ce8e6f70fa'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.749083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b721abd6554b389a3c309306dcf1a0', 'request_id': 'req-17084de9-1a17-45d3-9def-23ce8e6f70fa'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.777780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:35.797959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb6545cd0f840b5bdecb56172fdc5ea', 'request_id': 'req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.804864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb6545cd0f840b5bdecb56172fdc5ea', 'request_id': 'req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.815619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] The token cache is being invalidate because project 2fb6545cd0f840b5bdecb56172fdc5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:35.815969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.816614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7be3fa8-0a69-41d8-b5ae-38ab78c3ab96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.818135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 997/1971] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:35 2022] DELETE /identity/v3/projects/2fb6545cd0f840b5bdecb56172fdc5ea => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:35.819432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.819710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.819981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.840285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:35.842042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.857595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] The token cache is being invalidate because project 65b721abd6554b389a3c309306dcf1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:35.857907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.858562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-17084de9-1a17-45d3-9def-23ce8e6f70fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:35.860331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 975/1972] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:35 2022] DELETE /identity/v3/projects/65b721abd6554b389a3c309306dcf1a0 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:35.861396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.861681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.861952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.882306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:35.883588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.904344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 998/1973] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:35.905361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:35.906719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:35.927491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c192d9-92d0-47eb-b036-1b34c60e20c5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:35.933882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:35.935038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:35.935503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:35.935904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] PATH_INFO: `/v3/users/f52d5e7526cf485299d3858afb4decf3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:35.937666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52d5e7526cf485299d3858afb4decf3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:35.939998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:35.960093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 976/1974] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:35.960845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:35.962131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.020732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29c41-52ee-4670-92a3-8dd8a52a807b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.024382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.024997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.025275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.025548 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.035939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.037367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.047171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 977/1975] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:36.047969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.049298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.062771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0004b0a-a632-435e-8302-28737db4e450 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.067676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.068344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.068602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.069027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] PATH_INFO: `/v3/projects/d59d1454779e43e9b0f61c74d0047da4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.070160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59d1454779e43e9b0f61c74d0047da4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.072141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.128716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52d5e7526cf485299d3858afb4decf3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.129409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52d5e7526cf485299d3858afb4decf3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.130129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52d5e7526cf485299d3858afb4decf3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.137121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52d5e7526cf485299d3858afb4decf3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.152991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d62ffa56-378d-4fac-80e7-ea5751ff0b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52d5e7526cf485299d3858afb4decf3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.227026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59d1454779e43e9b0f61c74d0047da4', 'request_id': 'req-3f7ada92-60bb-4e72-ad74-24a30d266679'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.235025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59d1454779e43e9b0f61c74d0047da4', 'request_id': 'req-3f7ada92-60bb-4e72-ad74-24a30d266679'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.236655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:36.242527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59d1454779e43e9b0f61c74d0047da4', 'request_id': 'req-3f7ada92-60bb-4e72-ad74-24a30d266679'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.247437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 999/1976] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:35 2022] DELETE /identity/v3/users/f52d5e7526cf485299d3858afb4decf3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:36.248577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef9c36-044a-465f-9326-0941bd7741a1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.248863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef9c36-044a-465f-9326-0941bd7741a1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.249142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef9c36-044a-465f-9326-0941bd7741a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.277943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ebef9c36-044a-465f-9326-0941bd7741a1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:36.280314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ebef9c36-044a-465f-9326-0941bd7741a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.334702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] The token cache is being invalidate because project d59d1454779e43e9b0f61c74d0047da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:36.338758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.338758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3f7ada92-60bb-4e72-ad74-24a30d266679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.344287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 978/1977] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:36 2022] DELETE /identity/v3/projects/d59d1454779e43e9b0f61c74d0047da4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:36.347255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.347561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.347850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.382586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:36.382586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.390007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1000/1978] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:36.396662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 979/1979] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:36.398650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.402883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.403824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.404926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.416992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5fbb0-6ca4-467e-81de-b486183c7b1a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.426254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.428279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.428647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.430038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.430156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:36.432423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.434134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.443854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:36.445109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.455725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1001/1980] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:36.456148 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 980/1981] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:36.460554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.460686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.470331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0c5ce-e9bf-4db5-9c6c-acb5029822a6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.475529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5fc0e-27b5-4499-b98e-224de51aa186 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.476679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.479336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.479336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.479336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.479336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:36.479336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.480588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.480978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.481620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.481904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.482741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.483909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.485853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.492601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1002/1982] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:36.498112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.509983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77acfb-2d18-4df0-9b13-d846f6c52c98 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.515512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.516116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.516399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.516709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.517441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.519391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.618271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 981/1983] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:36.619365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.623032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.643640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1003/1984] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:36.644711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.663790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:36.713533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fa832-fe97-44b2-b5e4-94cae21326fd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.733461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432e2fc-0f5d-4168-b662-ee95f70123f1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.740262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.740875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.741149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.741416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.742047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.744383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.780544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.781226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.781576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.781924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] PATH_INFO: `/v3/projects/d093224fbda4411aa6ddae986e18b587` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.783105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d093224fbda4411aa6ddae986e18b587)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.785252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.844387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1004/1985] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:36.844387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.858269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a05775-786b-4ce2-9f85-b391cc3cfe0c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:36.863358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:36.863987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:36.864264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:36.864536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:36.867425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:36.867531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:36.933222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd093224fbda4411aa6ddae986e18b587', 'request_id': 'req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.940451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd093224fbda4411aa6ddae986e18b587', 'request_id': 'req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.947026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd093224fbda4411aa6ddae986e18b587', 'request_id': 'req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:36.953361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1005/1986] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:36 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:36.954186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:36.955812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:36.967980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d4381-4dea-496c-849a-1982a3739d3c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.024332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.024889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.025173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.025451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] PATH_INFO: `/v3/projects/74a6c7bfb1e7454caa9b9fc588b4eae2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.027044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a6c7bfb1e7454caa9b9fc588b4eae2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.029334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.066038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] The token cache is being invalidate because project d093224fbda4411aa6ddae986e18b587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:37.066334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.067159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.068677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 982/1987] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:36 2022] DELETE /identity/v3/projects/d093224fbda4411aa6ddae986e18b587 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:37.069503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.070789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.083224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec4d30-8cc1-43dd-b318-463e3ccec2c3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.087759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.088333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.088613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.088883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.089511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.091809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.101425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 983/1988] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:37.102203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.103750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.116289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7548-656e-45cf-a556-fbabe3d37267 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.120861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.121426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.121704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.125599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.125599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.125599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.134500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 984/1989] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:37.135504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.137106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.150301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca361c9e-77a2-4fab-86f7-a7c17d0f8075 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.159763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a6c7bfb1e7454caa9b9fc588b4eae2', 'request_id': 'req-6148a068-88be-4f0e-9793-8759eae86528'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.205748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.206376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.206686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.206960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] PATH_INFO: `/v3/users/f686591cef6f4f389175758127bae39f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.208095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f686591cef6f4f389175758127bae39f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.210068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.246032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a6c7bfb1e7454caa9b9fc588b4eae2', 'request_id': 'req-6148a068-88be-4f0e-9793-8759eae86528'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.255512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a6c7bfb1e7454caa9b9fc588b4eae2', 'request_id': 'req-6148a068-88be-4f0e-9793-8759eae86528'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.266317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] The token cache is being invalidate because project 74a6c7bfb1e7454caa9b9fc588b4eae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:37.266696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.267411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.268960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1006/1990] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:36 2022] DELETE /identity/v3/projects/74a6c7bfb1e7454caa9b9fc588b4eae2 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:37.269135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:37.269729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.271015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.283888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148a068-88be-4f0e-9793-8759eae86528 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.288445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.289060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.289343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.289607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.290229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.291820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.303283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1007/1991] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:37.304068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.305660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.320087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8ed0e-3055-4b9e-8edd-6a802481800a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.324671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971910eb-0873-492a-ac62-922c7b23804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.325283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971910eb-0873-492a-ac62-922c7b23804c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.325554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971910eb-0873-492a-ac62-922c7b23804c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.325826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971910eb-0873-492a-ac62-922c7b23804c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.326444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971910eb-0873-492a-ac62-922c7b23804c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.328013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971910eb-0873-492a-ac62-922c7b23804c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.337608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1008/1992] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:37.338751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.339035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.339301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.359113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:37.360382 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.368132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1009/1993] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:37.368854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.377835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec2c28-e757-4f6f-a158-05bc313ba71f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.382443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.383083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.383360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.383636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] PATH_INFO: `/v3/projects/34862427c3e1433faeb1cd387c92d25a/users/4a0ea37708334bae8d93eb95c6f7bfda/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.401896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f686591cef6f4f389175758127bae39f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.402606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f686591cef6f4f389175758127bae39f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.403788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f686591cef6f4f389175758127bae39f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.404295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34862427c3e1433faeb1cd387c92d25a, user_id=4a0ea37708334bae8d93eb95c6f7bfda, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.406955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.411710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f686591cef6f4f389175758127bae39f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.425164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fee97e08-cb4d-41b4-9536-cc4fea897443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f686591cef6f4f389175758127bae39f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:37.482406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1010/1994] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:37 2022] PUT /identity/v3/projects/34862427c3e1433faeb1cd387c92d25a/users/4a0ea37708334bae8d93eb95c6f7bfda/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:37.483569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.485307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.526008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 985/1995] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:37 2022] DELETE /identity/v3/users/f686591cef6f4f389175758127bae39f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:37.535971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c1772-5f1e-482e-810b-e8b49cffd6ad None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.535971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c1772-5f1e-482e-810b-e8b49cffd6ad None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.535971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c1772-5f1e-482e-810b-e8b49cffd6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.554496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e78c1772-5f1e-482e-810b-e8b49cffd6ad None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:37.556265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e78c1772-5f1e-482e-810b-e8b49cffd6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.571929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d58865-70da-445c-b702-182a87da1465 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.629443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 986/1996] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:37.633796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.634127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.634408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.650904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.651824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.652098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.652371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] PATH_INFO: `/v3/projects/90a6f5d2f39f40dba24f4d900bfb7647/users/e7c25ef7518446528d4e8441efc40eb7/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.664912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] MFA Rules not processed for user `4a0ea37708334bae8d93eb95c6f7bfda`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:37.666127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.667471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a6f5d2f39f40dba24f4d900bfb7647, user_id=e7c25ef7518446528d4e8441efc40eb7, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.670327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.730140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 987/1997] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:37.731014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.741188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa1496-c2d3-48c0-bb2c-23973d6ae95f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.745831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.746419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.746736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.747015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.747680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:37.747987 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.749644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.761246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 988/1998] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:37.762079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.764520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.778096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df2a00-0b84-4bf0-a1ef-7dc590725352 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.783294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0c8c0-5b79-4fae-b72d-157e1fa3a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:37.783879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c8c0-5b79-4fae-b72d-157e1fa3a41c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.784159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c8c0-5b79-4fae-b72d-157e1fa3a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.785785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c8c0-5b79-4fae-b72d-157e1fa3a41c admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.785785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0c8c0-5b79-4fae-b72d-157e1fa3a41c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:37.786963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0c8c0-5b79-4fae-b72d-157e1fa3a41c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:37.821062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1011/1999] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:37 2022] PUT /identity/v3/projects/90a6f5d2f39f40dba24f4d900bfb7647/users/e7c25ef7518446528d4e8441efc40eb7/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:37.822787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:37.881611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 989/2000] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:37.882920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:37.883205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:37.883615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:37.910573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041eda1-2586-40f8-ac26-fb9c7193eca2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:37.915371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] MFA Rules not processed for user `e7c25ef7518446528d4e8441efc40eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:37.916650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:37.949081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:37.999601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 990/2001] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:38.000691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.021283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.021899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.022184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.022520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.023220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.025119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.079348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a87a68-2835-4612-bb3a-8250e07d5ec6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.084394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.085044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.085323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.085592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.086269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.088241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.128791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1012/2002] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/projects => generated 381 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:38.128791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.128791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.188750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733842f9-7ce4-4b96-a336-c4bd1bb76259 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.191415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 991/2003] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:37 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:38.192263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.194737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.197362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.198277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.198808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.199277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.218951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203f2ef-2f46-4356-af1a-8f701485cd63 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.278124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.280142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.281420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.282180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.282487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.282801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] PATH_INFO: `/v3/projects/66bdde05739743f29d7abb4219fbebdb` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.283953 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bdde05739743f29d7abb4219fbebdb)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.285860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.289912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1013/2004] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:38.290731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.301496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bbb41-7db4-4ea0-982a-a0e871501190 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.306205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.306979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.307132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.307383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.308029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.310465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.341563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:38.404607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bdde05739743f29d7abb4219fbebdb', 'request_id': 'req-ae4206ba-a3f9-4812-871c-9c1396ff523d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:38.411406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bdde05739743f29d7abb4219fbebdb', 'request_id': 'req-ae4206ba-a3f9-4812-871c-9c1396ff523d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:38.417198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bdde05739743f29d7abb4219fbebdb', 'request_id': 'req-ae4206ba-a3f9-4812-871c-9c1396ff523d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:38.418138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1014/2005] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:38 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:38.419501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.430928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c404ddf-04b8-42ef-bf23-3b69b5dac7af admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.479053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:38.492113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.493155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.493647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.494022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.494700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.496339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.508468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1015/2006] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:38.509425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.519478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93205f-fcd2-4b52-976f-79854f53b573 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.524191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.524776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.525049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.525317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.525972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.527806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.539152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1016/2007] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:38.540141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.551812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e599-35ee-43f3-a3b3-2dc91ed466b4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.555722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] The token cache is being invalidate because project 66bdde05739743f29d7abb4219fbebdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:38.556327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:38.557180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:38.558196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaddf19-5495-4d1c-8155-aedc2bf960bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.558971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaddf19-5495-4d1c-8155-aedc2bf960bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.562307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 992/2008] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:38 2022] DELETE /identity/v3/projects/66bdde05739743f29d7abb4219fbebdb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:38.562601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaddf19-5495-4d1c-8155-aedc2bf960bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.562601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaddf19-5495-4d1c-8155-aedc2bf960bb admin admin] PATH_INFO: `/v3/projects/66ccd101d3b94010b06e0f59610aadc1/users/298c3b3eb021439584e2d35820540f6f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.562795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.562795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.575030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaddf19-5495-4d1c-8155-aedc2bf960bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ccd101d3b94010b06e0f59610aadc1, user_id=298c3b3eb021439584e2d35820540f6f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.577744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaddf19-5495-4d1c-8155-aedc2bf960bb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.581031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4206ba-a3f9-4812-871c-9c1396ff523d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.586424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc7f5-0206-45a1-9c64-437f967a932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.587085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc7f5-0206-45a1-9c64-437f967a932c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.587361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc7f5-0206-45a1-9c64-437f967a932c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.587629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc7f5-0206-45a1-9c64-437f967a932c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.588273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc7f5-0206-45a1-9c64-437f967a932c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.589845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc7f5-0206-45a1-9c64-437f967a932c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.600706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 993/2009] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:38.601881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.602161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.602476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.622563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:38.623757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.631817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 994/2010] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:38.632559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.642267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad679a-9e09-4ebc-87b8-56a03e249e34 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.648897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.648897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.648897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.648897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] PATH_INFO: `/v3/projects/ab4db535c36f4ccb8ac7df6ffb4240ac/users/9607e791e82741498a2b9cb22b2760d8/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.662274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4db535c36f4ccb8ac7df6ffb4240ac, user_id=9607e791e82741498a2b9cb22b2760d8, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.664998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.675641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1017/2011] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:38 2022] PUT /identity/v3/projects/66ccd101d3b94010b06e0f59610aadc1/users/298c3b3eb021439584e2d35820540f6f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:38.678765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.678765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.678765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.697574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:38.698785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.759072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1018/2012] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:38.759899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.761125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.772707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e396549-b8e7-48cb-ab5c-279ef8230d0c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.777786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.778384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.778699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.778972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.779601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.781219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.789411 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 995/2013] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:38 2022] PUT /identity/v3/projects/ab4db535c36f4ccb8ac7df6ffb4240ac/users/9607e791e82741498a2b9cb22b2760d8/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:38.790237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.802528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1019/2014] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:38.803432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.815016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc8792-0e0b-4fc8-a160-76925931ef6c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.863210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ae8b1-02a6-4126-b7e8-9b3305292ba5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.867995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.868592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.868869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.869140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.869780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:38.870081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.871259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.871705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c7ec3-bafa-4742-abdf-d7f1fd40abc2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.872537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.872817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.873095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] PATH_INFO: `/v3/projects/66ccd101d3b94010b06e0f59610aadc1/users/298c3b3eb021439584e2d35820540f6f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.875045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ccd101d3b94010b06e0f59610aadc1, user_id=298c3b3eb021439584e2d35820540f6f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.877700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.884876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 996/2015] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:38.886113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0c9f1-02eb-4501-a0ab-2de995ec45dd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.886396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0c9f1-02eb-4501-a0ab-2de995ec45dd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.886705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0c9f1-02eb-4501-a0ab-2de995ec45dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.888087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1020/2016] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:38 2022] PUT /identity/v3/projects/66ccd101d3b94010b06e0f59610aadc1/users/298c3b3eb021439584e2d35820540f6f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:38.890322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:38.915707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-acf0c9f1-02eb-4501-a0ab-2de995ec45dd None None] MFA Rules not processed for user `9607e791e82741498a2b9cb22b2760d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:38.916920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-acf0c9f1-02eb-4501-a0ab-2de995ec45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:38.957703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2477dd9-4d31-4e54-9ff4-1c6aef8af53e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:38.962515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:38.963152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.963429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.963702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:38.964333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:38.966992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:38.986813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 997/2017] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:38.986813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:38.986813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:38.986813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:38.987225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.014384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] MFA Rules not processed for user `298c3b3eb021439584e2d35820540f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:39.015740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:39.075979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:39.085047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1021/2018] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:38 2022] POST /identity/v3/projects => generated 355 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:39.089048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.094343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 998/2019] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:39.095407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.105371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1467e37-0e00-4b59-8b57-5c771a76827f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.166413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.167320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.167769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.169241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.169541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.171804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.175099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a421-9820-4482-b9ee-f8b884af5524 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.180140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.180949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.181425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.181808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.182598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.185239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.284787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 999/2020] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:39 2022] POST /identity/v3/users => generated 278 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:39.285961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.288671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:39.292389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1022/2021] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:39 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:39.292389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.365652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaf622-279f-45a9-9a7e-fd6b449fa927 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.376121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2714-27af-4b6c-b799-cd7dd0a37b66 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.381066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.381744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.382106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.382432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.383170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.384819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.395918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1023/2022] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:39.399559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.411023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a42aea-ac48-4617-b120-209cef83b74a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.416927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.417674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.417959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.418233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.419510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.421458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.425468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.426459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.426615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.427314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.429136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:39.493849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.495506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.506482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1000/2023] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:39.507268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.522419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ca43b-f57a-4d71-b988-42e8608088dd service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.530451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.530905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1024/2024] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:39 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:39.531532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.531904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.532230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.532876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.534493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.534747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.544703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7c0dc-1725-46ef-afc7-ed8550f2c139 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.548506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1001/2025] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:39.549580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.549919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.550618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.550962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.551339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.552046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.553829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.559194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9528-ed25-4ff5-bea9-87fa8e7bf95a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.564234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d603d7a-665b-4a07-807b-520c614e77ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.564881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d603d7a-665b-4a07-807b-520c614e77ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.565229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d603d7a-665b-4a07-807b-520c614e77ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.570682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d603d7a-665b-4a07-807b-520c614e77ef admin admin] PATH_INFO: `/v3/projects/0b5b2b40cde24cd8976f795fab6292b0/users/a99a78a868c54c44be83c061b12e759d/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.571044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1025/2026] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:39.573578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.583444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d603d7a-665b-4a07-807b-520c614e77ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5b2b40cde24cd8976f795fab6292b0, user_id=a99a78a868c54c44be83c061b12e759d, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.584698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f167d7b-0ed0-4c26-8b9e-82efba66aa99 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.586445 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d603d7a-665b-4a07-807b-520c614e77ef admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.591358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.591874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:39.592172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.592587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.592779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] PATH_INFO: `/v3/projects/459a2834dcb7495da894062470ccaa2b/users/2972712dd3e7468c9136d02a1a49d5e2/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.608135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=459a2834dcb7495da894062470ccaa2b, user_id=2972712dd3e7468c9136d02a1a49d5e2, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.611020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.686676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1002/2027] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:39 2022] PUT /identity/v3/projects/0b5b2b40cde24cd8976f795fab6292b0/users/a99a78a868c54c44be83c061b12e759d/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:39.698374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.698692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.698970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.714857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1026/2028] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:39 2022] PUT /identity/v3/projects/459a2834dcb7495da894062470ccaa2b/users/2972712dd3e7468c9136d02a1a49d5e2/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:39.722626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.727812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] MFA Rules not processed for user `a99a78a868c54c44be83c061b12e759d`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:39.729174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:39.787105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059808c3-3f29-42d6-a16c-8c820f2ddb8f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.791838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.792444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.792774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.793053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.794307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.795943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.810631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1003/2029] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:39.810922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1027/2030] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:39.818803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.819645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:39.830748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c23f6-c64d-451f-84e8-06a9151eba53 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.836618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbbe25-489d-43a6-a75d-21e562012f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.837555 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbbe25-489d-43a6-a75d-21e562012f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.837971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbbe25-489d-43a6-a75d-21e562012f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.838380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbbe25-489d-43a6-a75d-21e562012f44 admin admin] PATH_INFO: `/v3/projects/459a2834dcb7495da894062470ccaa2b/users/2972712dd3e7468c9136d02a1a49d5e2/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.840327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40806b7-260f-4fb3-9c85-b1fc233b38f6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:39.841102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbbe25-489d-43a6-a75d-21e562012f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459a2834dcb7495da894062470ccaa2b, user_id=2972712dd3e7468c9136d02a1a49d5e2, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.844420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbbe25-489d-43a6-a75d-21e562012f44 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.854705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:39.855415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.855913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.856805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.857894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:39.867299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1028/2031] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:39 2022] PUT /identity/v3/projects/459a2834dcb7495da894062470ccaa2b/users/2972712dd3e7468c9136d02a1a49d5e2/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:39.867477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:39.869237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:39.869611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:39.869940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:39.910940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] MFA Rules not processed for user `2972712dd3e7468c9136d02a1a49d5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:39.912219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:39.931814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:39.990388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1029/2032] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:39.995887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1004/2033] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:39 2022] POST /identity/v3/projects => generated 357 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:40.002371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.025325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.088375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:40.088534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057e1c7-8360-4a42-8682-244595b96183 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.130250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2280ac-1e48-4654-b513-a6933bd16a28 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.130250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.130250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.130250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.130620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.151998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.152731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.153003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.158605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.158605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.158605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.178991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.180521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.190616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1005/2034] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:40.241473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1030/2035] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:40 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:40.245620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.256652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fc861-a3a1-487a-912e-9b506cf70807 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.261136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.261745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.262846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.263360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.264418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.266088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.276218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1006/2036] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:40.281631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.291752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c494454-35c9-4378-98f7-9295a8608110 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.296409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.297057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.297336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.297611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] PATH_INFO: `/v3/projects/deada32cf59e47208dcd1c55ab47af29/users/6454972e80134e7785ce97417fd9f03b/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.312723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deada32cf59e47208dcd1c55ab47af29, user_id=6454972e80134e7785ce97417fd9f03b, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.315420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.328742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:40.395351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1031/2037] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:40 2022] PUT /identity/v3/projects/deada32cf59e47208dcd1c55ab47af29/users/6454972e80134e7785ce97417fd9f03b/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:40.403637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.421330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec58146-a36e-4a47-9ef6-8938d5bc3f9e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.478090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa99df5-02d9-49cf-8108-e8686f95b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.478722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa99df5-02d9-49cf-8108-e8686f95b6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.478997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa99df5-02d9-49cf-8108-e8686f95b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.479265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa99df5-02d9-49cf-8108-e8686f95b6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.479905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa99df5-02d9-49cf-8108-e8686f95b6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.486400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa99df5-02d9-49cf-8108-e8686f95b6ce admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.502717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1007/2038] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:40.506052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.517549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26821b-720e-4955-a792-9a58f8b24d33 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.531836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.532517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.532794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.533117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] PATH_INFO: `/v3/projects/deada32cf59e47208dcd1c55ab47af29/users/6454972e80134e7785ce97417fd9f03b/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.544519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] RBAC: Authorizing `identity:create_grant(project_id=deada32cf59e47208dcd1c55ab47af29, user_id=6454972e80134e7785ce97417fd9f03b, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.544519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.557330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1032/2039] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:40 2022] PUT /identity/v3/projects/deada32cf59e47208dcd1c55ab47af29/users/6454972e80134e7785ce97417fd9f03b/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:40.568269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.568695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.568830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.611678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] MFA Rules not processed for user `6454972e80134e7785ce97417fd9f03b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:40.611678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:40.719466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:40.751958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1008/2040] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:40.766213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:40.874452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:40.941916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84dca38-10eb-44c6-a584-d349eb56749b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:40.945896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:40.946954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:40.947067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:40.947366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:40.961468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:40.962988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:40.975388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1033/2041] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:41.345597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:41.359598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8b0c2-57c4-4561-a3b9-b19ef740064d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:41.363661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:41.364595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:41.366886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:41.367372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:41.434988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:41.462294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:41.462294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:41.471244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1009/2042] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:41.839188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:41.842585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:41.930086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d346c0b-5dfa-4a2c-a3bd-ba6d8b36f8cc service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:41.935929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:41.936574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:41.936854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:41.937132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] PATH_INFO: `/v3/users/fdd4a06dacd541f9949af0c4cd1ddfed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:41.938285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd4a06dacd541f9949af0c4cd1ddfed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:41.945776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:41.975621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:41.990792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a111f-0d99-49d2-8da3-8a4a645778ec service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:41.997235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:41.998356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:41.999171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:41.999626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] PATH_INFO: `/v3/users/4a0ea37708334bae8d93eb95c6f7bfda` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.000737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0ea37708334bae8d93eb95c6f7bfda)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:42.003929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:42.172049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd4a06dacd541f9949af0c4cd1ddfed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.172698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd4a06dacd541f9949af0c4cd1ddfed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.173317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd4a06dacd541f9949af0c4cd1ddfed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.180473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd4a06dacd541f9949af0c4cd1ddfed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.193409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd4a06dacd541f9949af0c4cd1ddfed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.202406 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0ea37708334bae8d93eb95c6f7bfda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.203213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0ea37708334bae8d93eb95c6f7bfda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.203954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0ea37708334bae8d93eb95c6f7bfda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.211314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0ea37708334bae8d93eb95c6f7bfda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.223889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2cfb845-e945-4d49-9ce7-566f7e6bb457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0ea37708334bae8d93eb95c6f7bfda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.270910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1034/2043] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:41 2022] DELETE /identity/v3/users/fdd4a06dacd541f9949af0c4cd1ddfed => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:42.270910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:42.270910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.281659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:42.335787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c7fec-0ca9-4977-8398-f9a6f1f9389a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:42.340992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:42.341621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.341894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.342163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] PATH_INFO: `/v3/users/9607e791e82741498a2b9cb22b2760d8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.343355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9607e791e82741498a2b9cb22b2760d8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:42.345349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:42.348686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:42.364076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1010/2044] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:41 2022] DELETE /identity/v3/users/4a0ea37708334bae8d93eb95c6f7bfda => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:42.365565 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e244-b886-4d92-969a-6f0be159f96c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.366032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e244-b886-4d92-969a-6f0be159f96c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.366310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e244-b886-4d92-969a-6f0be159f96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.386682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3622e244-b886-4d92-969a-6f0be159f96c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:42.387995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3622e244-b886-4d92-969a-6f0be159f96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.448628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1011/2045] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:42.450395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.450728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.451138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.471095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:42.472355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.480516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1012/2046] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:42.481304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:42.482522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.540167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123958d7-a306-4b1f-ba10-9822e1764a97 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:42.544754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:42.545358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.545630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.545897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.556036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:42.557617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:42.558915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9607e791e82741498a2b9cb22b2760d8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.560634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9607e791e82741498a2b9cb22b2760d8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.560934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9607e791e82741498a2b9cb22b2760d8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.568372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1013/2047] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:42.568516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9607e791e82741498a2b9cb22b2760d8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.569184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:42.581463 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93d268-0a9b-4d5a-b4fa-6db0a33662d2 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:42.585578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:42.586177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.586477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.586805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.588872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.588987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9607e791e82741498a2b9cb22b2760d8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:42.605140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:42.607535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512247e6-c303-4c5d-9b47-f72120395391 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:42.617490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1014/2048] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:42.692708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1035/2049] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:42 2022] DELETE /identity/v3/users/9607e791e82741498a2b9cb22b2760d8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:42.697094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.697386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.697667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.720315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:42.721794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.787498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1015/2050] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:42.797246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:42.798825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:42.861191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10145-53ae-49af-b5e9-75f5b2bb8aa2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:42.865112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:42.865803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:42.866151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:42.866559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:42.877473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:42.878991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:42.892987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1036/2051] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:43.001340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:43.324169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:43.326707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:43.338870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcbe1b-94bb-4681-a25d-d74165718c34 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:43.344465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:43.345096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:43.345378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:43.345655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] PATH_INFO: `/v3/projects/34862427c3e1433faeb1cd387c92d25a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:43.346938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34862427c3e1433faeb1cd387c92d25a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:43.348888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:43.358383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:43.360457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:43.383385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98598877-ca89-429c-8fbe-14fd58b214ac service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:43.397306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:43.399033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:43.399219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:43.399718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] PATH_INFO: `/v3/projects/f8dda6c4980d469e97d63f34ba988ef6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:43.401003 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dda6c4980d469e97d63f34ba988ef6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:43.403896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:43.538401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:43.540665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34862427c3e1433faeb1cd387c92d25a', 'request_id': 'req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.549592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34862427c3e1433faeb1cd387c92d25a', 'request_id': 'req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.569378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dda6c4980d469e97d63f34ba988ef6', 'request_id': 'req-66012b44-e295-48e5-9941-c4cb2df0c08e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.582643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34862427c3e1433faeb1cd387c92d25a', 'request_id': 'req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.645748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dda6c4980d469e97d63f34ba988ef6', 'request_id': 'req-66012b44-e295-48e5-9941-c4cb2df0c08e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.653342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dda6c4980d469e97d63f34ba988ef6', 'request_id': 'req-66012b44-e295-48e5-9941-c4cb2df0c08e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.664007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] The token cache is being invalidate because project f8dda6c4980d469e97d63f34ba988ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:43.664299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.664952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.666560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1037/2052] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:43 2022] DELETE /identity/v3/projects/f8dda6c4980d469e97d63f34ba988ef6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:43.667396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:43.669422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:43.734820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66012b44-e295-48e5-9941-c4cb2df0c08e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:43.739490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:43.740115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:43.740394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:43.740666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] PATH_INFO: `/v3/projects/ab4db535c36f4ccb8ac7df6ffb4240ac` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:43.741815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4db535c36f4ccb8ac7df6ffb4240ac)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:43.743769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:43.748689 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] The token cache is being invalidate because project 34862427c3e1433faeb1cd387c92d25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:43.758710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.758710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.758710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1016/2053] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:43 2022] DELETE /identity/v3/projects/34862427c3e1433faeb1cd387c92d25a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:43.758710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:43.758710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:43.818195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfe80-9c78-47ca-bd3f-193a61c69e19 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:43.823010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:43.823667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:43.823948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:43.824272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] PATH_INFO: `/v3/users/e7c25ef7518446528d4e8441efc40eb7` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:43.826083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c25ef7518446528d4e8441efc40eb7)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:43.828101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:43.849693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4db535c36f4ccb8ac7df6ffb4240ac', 'request_id': 'req-993089f4-9044-4ceb-bb7f-dccaee00e9c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.943523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4db535c36f4ccb8ac7df6ffb4240ac', 'request_id': 'req-993089f4-9044-4ceb-bb7f-dccaee00e9c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.950367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4db535c36f4ccb8ac7df6ffb4240ac', 'request_id': 'req-993089f4-9044-4ceb-bb7f-dccaee00e9c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.960773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] The token cache is being invalidate because project ab4db535c36f4ccb8ac7df6ffb4240ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:43.961063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.961712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-993089f4-9044-4ceb-bb7f-dccaee00e9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:43.963311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1038/2054] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:43 2022] DELETE /identity/v3/projects/ab4db535c36f4ccb8ac7df6ffb4240ac => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:43.967558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:43.967558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:43.967558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:43.985964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:43.987220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.044924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1039/2055] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:44.045747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:44.046996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.058423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0088-523c-4d4c-93d9-802a25ea020a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:44.063664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:44.064306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.064727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.065021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] PATH_INFO: `/v3/users/298c3b3eb021439584e2d35820540f6f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.066114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298c3b3eb021439584e2d35820540f6f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:44.068076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:44.098980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c25ef7518446528d4e8441efc40eb7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.099629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c25ef7518446528d4e8441efc40eb7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.100260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c25ef7518446528d4e8441efc40eb7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.107778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c25ef7518446528d4e8441efc40eb7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.119385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c25ef7518446528d4e8441efc40eb7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.199753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1017/2056] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:43 2022] DELETE /identity/v3/users/e7c25ef7518446528d4e8441efc40eb7 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:44.200590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:44.201931 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.262308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298c3b3eb021439584e2d35820540f6f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.263206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298c3b3eb021439584e2d35820540f6f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.263919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298c3b3eb021439584e2d35820540f6f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.273288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298c3b3eb021439584e2d35820540f6f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.279608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e03f7-a7cd-4f24-8d88-6257bb5a91c3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:44.285820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:44.286457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.286917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.287295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] PATH_INFO: `/v3/users/2972712dd3e7468c9136d02a1a49d5e2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.290711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2972712dd3e7468c9136d02a1a49d5e2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:44.290832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4a756753-a122-4aa7-bfe9-f15d7fb703b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298c3b3eb021439584e2d35820540f6f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.290929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:44.363950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:44.411007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1040/2057] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:44 2022] DELETE /identity/v3/users/298c3b3eb021439584e2d35820540f6f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:44.412547 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd604c0-e606-46c8-a262-c6ca88251ec1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.412856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd604c0-e606-46c8-a262-c6ca88251ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.413135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd604c0-e606-46c8-a262-c6ca88251ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.437143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-abd604c0-e606-46c8-a262-c6ca88251ec1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:44.438539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-abd604c0-e606-46c8-a262-c6ca88251ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.505741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2972712dd3e7468c9136d02a1a49d5e2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.506378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2972712dd3e7468c9136d02a1a49d5e2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.507228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2972712dd3e7468c9136d02a1a49d5e2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.511475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1041/2058] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:44.511475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.511475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.511475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.520413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2972712dd3e7468c9136d02a1a49d5e2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.531906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:44.533128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.538105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2972712dd3e7468c9136d02a1a49d5e2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:44.540990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1042/2059] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:44.548268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:44.549838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.564704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f9a9-e0fb-48da-89cc-7417c799ae5d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:44.571015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:44.571952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.572327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.572670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.573387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:44.573752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:44.575544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:44.587904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1043/2060] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:44.588857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:44.592862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:44.634963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1018/2061] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:44 2022] DELETE /identity/v3/users/2972712dd3e7468c9136d02a1a49d5e2 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:44.634963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:44.662359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:44.681687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785e557-c2ba-465f-9e7f-c42f314c1893 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:44.733320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:44.763936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcb963-318e-45b1-a85a-feeb69b37e53 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:44.778635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:44.778635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.778635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.778635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.779343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:44.779343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:44.799296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:44.799971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:44.800255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:44.800521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:44.814194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:44.815774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:44.828060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1044/2062] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:44.924979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1019/2063] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:44 2022] POST /identity/v3/projects => generated 359 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:44.925956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:44.940464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842af03-848f-43f1-be35-a06c9a6a617b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.008792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.012573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.012573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.012573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.012573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.014436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.102432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.104698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.119428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a331e-fe0d-4d6e-9b3c-203b47ef7b83 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.138022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1045/2064] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:44 2022] POST /identity/v3/users => generated 280 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:45.138374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.139066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.145160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.145561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.147757 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] PATH_INFO: `/v3/projects/66ccd101d3b94010b06e0f59610aadc1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.150654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ccd101d3b94010b06e0f59610aadc1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.152730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecbc67-91b3-497a-a69f-dd0f74bf1d10 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.152961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.157903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.158725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.159105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.159483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.160406 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.162487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.175741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1046/2065] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:45.181047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.193293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a0a2d-0d24-4795-8987-b95c0abb6407 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.198828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.200115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.200401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.200676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.201324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.205936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.214449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1047/2066] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:45.219834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.251436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee92b72-9f20-46ec-967a-625970d19133 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.263184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.263967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.264314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.264674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] PATH_INFO: `/v3/projects/6e38511b5aaf4541a753db7e8199b7d8/users/71c1bf89a3754bfda53c50df7aeb5285/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.280920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e38511b5aaf4541a753db7e8199b7d8, user_id=71c1bf89a3754bfda53c50df7aeb5285, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.283924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.362261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1048/2067] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:45 2022] PUT /identity/v3/projects/6e38511b5aaf4541a753db7e8199b7d8/users/71c1bf89a3754bfda53c50df7aeb5285/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:45.372119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.372119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.390394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ccd101d3b94010b06e0f59610aadc1', 'request_id': 'req-ba37cde4-485e-4c2b-a926-a3b52c69b61a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.406705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ccd101d3b94010b06e0f59610aadc1', 'request_id': 'req-ba37cde4-485e-4c2b-a926-a3b52c69b61a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.420645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ccd101d3b94010b06e0f59610aadc1', 'request_id': 'req-ba37cde4-485e-4c2b-a926-a3b52c69b61a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.439514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd525fc-1850-49b4-b9b8-cd7002b8a9c5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.481202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:45.499029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.499660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.499935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.500203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] PATH_INFO: `/v3/projects/459a2834dcb7495da894062470ccaa2b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.501350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459a2834dcb7495da894062470ccaa2b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.503257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.534874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] The token cache is being invalidate because project 66ccd101d3b94010b06e0f59610aadc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:45.535160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.535770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ba37cde4-485e-4c2b-a926-a3b52c69b61a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.537248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1020/2068] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:45 2022] DELETE /identity/v3/projects/66ccd101d3b94010b06e0f59610aadc1 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:45.538479 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.538781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.539384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.567864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] MFA Rules not processed for user `71c1bf89a3754bfda53c50df7aeb5285`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:45.569064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.605338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459a2834dcb7495da894062470ccaa2b', 'request_id': 'req-ea75ec7f-9451-4b80-b621-c149ba89cee1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.657173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1021/2069] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:45.657173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.657173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.673574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ad2b8-ca9d-4479-928e-6eae1d752900 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.702918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459a2834dcb7495da894062470ccaa2b', 'request_id': 'req-ea75ec7f-9451-4b80-b621-c149ba89cee1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.706107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:45.712097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459a2834dcb7495da894062470ccaa2b', 'request_id': 'req-ea75ec7f-9451-4b80-b621-c149ba89cee1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.733779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] The token cache is being invalidate because project 459a2834dcb7495da894062470ccaa2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:45.734512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.736615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ea75ec7f-9451-4b80-b621-c149ba89cee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.744276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1049/2070] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:45 2022] DELETE /identity/v3/projects/459a2834dcb7495da894062470ccaa2b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:45.744276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.749955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.749955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.770901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.772624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.772624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.772624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] PATH_INFO: `/v3/projects/90a6f5d2f39f40dba24f4d900bfb7647` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.773648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a6f5d2f39f40dba24f4d900bfb7647)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:45.775918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:45.778405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:45.779710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.796399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1050/2071] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:45.797909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:45.799730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.891604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb4036-4512-458a-9214-6f7227c25920 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:45.899531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a6f5d2f39f40dba24f4d900bfb7647', 'request_id': 'req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.906898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a6f5d2f39f40dba24f4d900bfb7647', 'request_id': 'req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.915298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a6f5d2f39f40dba24f4d900bfb7647', 'request_id': 'req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.943469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:45.944070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.944348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.944613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:45.946400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:45.987087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] The token cache is being invalidate because project 90a6f5d2f39f40dba24f4d900bfb7647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:45.987766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.988801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d1fc1322-68e2-49d5-b7b7-1532c603a3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:45.990627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1022/2072] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:45 2022] DELETE /identity/v3/projects/90a6f5d2f39f40dba24f4d900bfb7647 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:45.992230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77d1f7-330f-47b2-9061-3fc0bd3603e8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:45.992709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77d1f7-330f-47b2-9061-3fc0bd3603e8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:45.993277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77d1f7-330f-47b2-9061-3fc0bd3603e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.015346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5e77d1f7-330f-47b2-9061-3fc0bd3603e8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:46.016684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5e77d1f7-330f-47b2-9061-3fc0bd3603e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.017098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.019257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1598f-1a9f-4b39-984e-d54d2d23046e service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.031441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1051/2073] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:46.033183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f672531-fde1-432b-b7fa-5e3c2836b0ee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.033470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f672531-fde1-432b-b7fa-5e3c2836b0ee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.033740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f672531-fde1-432b-b7fa-5e3c2836b0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.054563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f672531-fde1-432b-b7fa-5e3c2836b0ee None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:46.056084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f672531-fde1-432b-b7fa-5e3c2836b0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.092904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1023/2074] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:46.094158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.094484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.094753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.117301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:46.118513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.126533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:46.128390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1024/2075] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:46.129087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.130688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.141733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1052/2076] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:46.143281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e0ccb-ccda-45cd-8e60-efe51fbf7e42 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.144033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a4118-d98d-4414-99be-f89c2b84902f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.144413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a4118-d98d-4414-99be-f89c2b84902f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.144953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a4118-d98d-4414-99be-f89c2b84902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.151945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.152663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.153127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.153400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] PATH_INFO: `/v3/users/71c1bf89a3754bfda53c50df7aeb5285` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.154614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c1bf89a3754bfda53c50df7aeb5285)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.156806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.165558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-df9a4118-d98d-4414-99be-f89c2b84902f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:46.166862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df9a4118-d98d-4414-99be-f89c2b84902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.175525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1053/2077] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:46.176835 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.177116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.177383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.198098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:46.199394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.207902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1054/2078] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:46.208499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.218251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a987eb-6f05-406d-a944-ad545e2d22b7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.223075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.223669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.223948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.224223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.224938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:46.225246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.226884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.241483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1055/2079] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:46.242544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.253322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7cb75-bd5f-4f47-898d-f78a0aeb9d74 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.258326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.258951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.259232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.259508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.260151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:46.260453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.262799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.266787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:46.277308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1056/2080] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:46.279465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.289939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205ecb-870f-470c-aa2c-72af3dc75da9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.294827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.295574 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.295953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.296349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.297302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:46.297710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.299560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.312704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1057/2081] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:46.313624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.326806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6354bc0-9c3f-44c7-8c5b-47f40af2f9d4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.333945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.334570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.334860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.335136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.335781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.337675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.345493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c1bf89a3754bfda53c50df7aeb5285'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:46.346164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c1bf89a3754bfda53c50df7aeb5285'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:46.346819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c1bf89a3754bfda53c50df7aeb5285'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:46.354714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c1bf89a3754bfda53c50df7aeb5285'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:46.367518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c1bf89a3754bfda53c50df7aeb5285'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:46.470083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1025/2082] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:46 2022] DELETE /identity/v3/users/71c1bf89a3754bfda53c50df7aeb5285 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:46.471456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.474361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1058/2083] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:46.474646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.475320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.478091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.545562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a540-0e8e-44ea-9665-d32516b05f76 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.549853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a7b06-b510-4989-a3f6-404aa1c86dcb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.551113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeda97f-c3b0-453b-a3ab-235433889942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.551655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda97f-c3b0-453b-a3ab-235433889942 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.551936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda97f-c3b0-453b-a3ab-235433889942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.552209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda97f-c3b0-453b-a3ab-235433889942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.552880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeda97f-c3b0-453b-a3ab-235433889942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.552980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:46.555488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeda97f-c3b0-453b-a3ab-235433889942 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.557616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.560135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.560135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.560135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.560391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.562374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.653695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1026/2084] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:46.655035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.655329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.655684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.666957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1059/2085] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/projects => generated 371 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:46.668714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.670587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.677007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:46.678336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.752035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054b1a5-12e3-477e-9db1-61531660cb08 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.757354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.758149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.758495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.758903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.759645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.761868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.762692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1027/2086] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:46.763849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.782684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b2d5f-449c-439a-a8b1-2479317636ca None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.788805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.789414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.789690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.789960 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.790682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.792942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.800418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:46.884743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1028/2087] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:46.884743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.885229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1060/2088] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:46.885229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:46.889517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:46.901552 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a13e49-0dcd-4276-be59-5b03b1c42366 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.915244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.916111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.916800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.917168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.918390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.923793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.968635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a070ef8-52b9-4510-aec3-72a44e738a59 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:46.972422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:46.973204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:46.973616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:46.974032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:46.992066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:46.992066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:46.998442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1061/2089] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:47.000927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.018218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510965-0fb4-42ec-b239-b99275b9b52f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.021043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1029/2090] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:46 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:47.021932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.025359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.026068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.026738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.027089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.028427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.039246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.047584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cccc8-a1d0-45fd-a99a-8aea32da2190 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.058709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1062/2091] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:47.060225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.062196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.062850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.063132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.063403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.064050 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.065650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.079539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786f68b-8b99-462e-af66-2574955b70d4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.084641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.085890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.086038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.086247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.086963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.088831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.106174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1063/2092] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:47.106174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.119513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1030/2093] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:47.120025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.130360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:47.142009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59049f8-1285-400c-b57a-a72ba22c5232 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.157317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.157794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8c78-dd0f-462d-8dbd-6a32fc88a5f2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.157932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.162939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.162939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.162939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.162939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.163311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.163653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.163935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.164216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.164876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.166673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.172627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1064/2094] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:47.173690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.181303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1031/2095] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:47.182115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.185080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc19e5-bcdf-4071-b16f-77080fa9777c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.189691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.190284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.190576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.190856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.191494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.193079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.195153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3604c86-7941-4bea-ad56-e17fe0f349c9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.200093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.200749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.201028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.201305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] PATH_INFO: `/v3/projects/3aa3291c0f334fc481f3e9027575d441/users/df80328a98704b4384eaac350c0e9b4d/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.203910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1065/2096] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:47.205311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.217876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa3291c0f334fc481f3e9027575d441, user_id=df80328a98704b4384eaac350c0e9b4d, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.220886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77228a4-92e8-4c42-af06-b1d72c6e98f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.221113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.226749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b8410-3b6e-404c-a472-ab5e7f316dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.227456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b8410-3b6e-404c-a472-ab5e7f316dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.227732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b8410-3b6e-404c-a472-ab5e7f316dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.228029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b8410-3b6e-404c-a472-ab5e7f316dbf admin admin] PATH_INFO: `/v3/projects/5482356a0cf74dc995decac42546f883/users/4eec5bc744b54cac82f94ea84ec90eeb/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.270108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b8410-3b6e-404c-a472-ab5e7f316dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5482356a0cf74dc995decac42546f883, user_id=4eec5bc744b54cac82f94ea84ec90eeb, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.273527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b8410-3b6e-404c-a472-ab5e7f316dbf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.348470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1032/2097] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:47 2022] PUT /identity/v3/projects/3aa3291c0f334fc481f3e9027575d441/users/df80328a98704b4384eaac350c0e9b4d/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:47.352196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.389965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1066/2098] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:47 2022] PUT /identity/v3/projects/5482356a0cf74dc995decac42546f883/users/4eec5bc744b54cac82f94ea84ec90eeb/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:47.391729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2841e-1a3c-4228-96c9-641c835ffb66 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.392178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2841e-1a3c-4228-96c9-641c835ffb66 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.392462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2841e-1a3c-4228-96c9-641c835ffb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.431710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bcf2841e-1a3c-4228-96c9-641c835ffb66 None None] MFA Rules not processed for user `df80328a98704b4384eaac350c0e9b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:47.432515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c2d-0f82-4e7a-8782-ff5ad286353e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.434056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bcf2841e-1a3c-4228-96c9-641c835ffb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:47.484357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:47.505530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.506343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.506471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.506780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] PATH_INFO: `/v3/projects/f219be7ffd74405fa8041e17b4e43b0b/users/4dc2e708ea964b11ab4cc4df205bfb91/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.523674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f219be7ffd74405fa8041e17b4e43b0b, user_id=4dc2e708ea964b11ab4cc4df205bfb91, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.527291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.541224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1067/2099] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:47.544499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19773-ad53-4fb3-913f-b79bc62e5471 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.544815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19773-ad53-4fb3-913f-b79bc62e5471 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.545252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19773-ad53-4fb3-913f-b79bc62e5471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.593038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ffd19773-ad53-4fb3-913f-b79bc62e5471 None None] MFA Rules not processed for user `4eec5bc744b54cac82f94ea84ec90eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:47.594409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ffd19773-ad53-4fb3-913f-b79bc62e5471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:47.640236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1033/2100] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:47 2022] PUT /identity/v3/projects/f219be7ffd74405fa8041e17b4e43b0b/users/4dc2e708ea964b11ab4cc4df205bfb91/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:47.641524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.677571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1068/2101] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:47.678926 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.679213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.679488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.692954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:47.719109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] MFA Rules not processed for user `4dc2e708ea964b11ab4cc4df205bfb91`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:47.720302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8e0b-7ae4-4528-bc36-4bea80bd5611 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.720451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:47.728089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.729000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.729513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.730002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.731045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.734167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.791144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1069/2102] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:47.791855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.795858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:47.845652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1034/2103] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:47 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:47.846905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:47.849713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:47.858156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6274c-0b12-42b5-8ba6-b20e73125cde None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.914944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.915686 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.915993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.916263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.930842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0d5f4-130c-4bec-85ac-a848c629cf61 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:47.936113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:47.936914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:47.937298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:47.937673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] PATH_INFO: `/v3/projects/6e38511b5aaf4541a753db7e8199b7d8` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:47.939018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e38511b5aaf4541a753db7e8199b7d8)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.941070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.983890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:47.985569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:47.995320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1070/2104] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:47.997920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.009295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2e5b-6a0c-4ada-8d8d-845b699e49e2 service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.013231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:48.015674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.016281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.016556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.016896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.018737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.019626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.063789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e38511b5aaf4541a753db7e8199b7d8', 'request_id': 'req-78de1710-a44e-45c6-9d38-7583cfb0dac6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:48.071325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e38511b5aaf4541a753db7e8199b7d8', 'request_id': 'req-78de1710-a44e-45c6-9d38-7583cfb0dac6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:48.079892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e38511b5aaf4541a753db7e8199b7d8', 'request_id': 'req-78de1710-a44e-45c6-9d38-7583cfb0dac6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:48.112053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1071/2105] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:47 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:48.115902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.195299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] The token cache is being invalidate because project 6e38511b5aaf4541a753db7e8199b7d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:48.195784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:48.196856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:48.200547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1035/2106] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:47 2022] DELETE /identity/v3/projects/6e38511b5aaf4541a753db7e8199b7d8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:48.201433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.207111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:48.270498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370f18-4968-4ba8-9b45-2823a6ca8f59 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.285713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a4b3-b64e-465a-a82d-c80fc562ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.286747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a4b3-b64e-465a-a82d-c80fc562ea47 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.287128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a4b3-b64e-465a-a82d-c80fc562ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.287503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a4b3-b64e-465a-a82d-c80fc562ea47 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.288297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4a4b3-b64e-465a-a82d-c80fc562ea47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.293159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4a4b3-b64e-465a-a82d-c80fc562ea47 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.302674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:48.345214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de1710-a44e-45c6-9d38-7583cfb0dac6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.352733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.353577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.354034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.354465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.355340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.358008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.430692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1072/2107] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:48 2022] POST /identity/v3/users => generated 298 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:48.432403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.432930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.434287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.457216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1036/2108] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:48 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:48.459619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.459828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:48.459828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:48.460918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:48.482630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1073/2109] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:48.482630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.489645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab851ce-af83-4621-bd93-509fc4a05433 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.495721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9618-c2fa-4a55-ae4d-d6defeacaeb3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.498318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.498972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.501691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a87d0-8fa0-4f1a-be8f-d5ba880efcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.502786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.503103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.503788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a87d0-8fa0-4f1a-be8f-d5ba880efcdf admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.503919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.504246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a87d0-8fa0-4f1a-be8f-d5ba880efcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.504614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a87d0-8fa0-4f1a-be8f-d5ba880efcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.505377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a87d0-8fa0-4f1a-be8f-d5ba880efcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.505501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.507332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a87d0-8fa0-4f1a-be8f-d5ba880efcdf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.518681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1037/2110] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:48.520523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.520644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.520951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.534628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1074/2111] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:48.534628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.546852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914992b-2e36-4154-88f4-6830a59297f9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.550359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:48.551846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:48.555933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.556602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.557333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.557600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] PATH_INFO: `/v3/projects/dfc3b58c039d4dd59467c0f10128573c/users/d29dabdeaf5a461b936a934b315d101d/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.567071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1038/2112] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:48.567071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.581938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28e99-0a96-4c97-b6bf-9fd3085a5bed None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.582237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc3b58c039d4dd59467c0f10128573c, user_id=d29dabdeaf5a461b936a934b315d101d, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.586095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.587725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.588392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.588672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.588964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] PATH_INFO: `/v3/projects/392829920ff54b09bdd2c75934153dfe/users/7ce2f809ba444e2e953abaf6a0b576c3/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.611022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392829920ff54b09bdd2c75934153dfe, user_id=7ce2f809ba444e2e953abaf6a0b576c3, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.614489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.696315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1075/2113] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:48 2022] PUT /identity/v3/projects/dfc3b58c039d4dd59467c0f10128573c/users/d29dabdeaf5a461b936a934b315d101d/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:48.699338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.754955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1039/2114] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:48 2022] PUT /identity/v3/projects/392829920ff54b09bdd2c75934153dfe/users/7ce2f809ba444e2e953abaf6a0b576c3/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:48.755576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.762766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecab30-4e07-4887-b042-e9b2a0e0c7e5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.831765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:48.843796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.844752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.846709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.847380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.848210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:48.849399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.854584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189a050-7769-4a14-acb7-01d66c5fdbe7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.856437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.861281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.861898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.862176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.862448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.863476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.865052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.871669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1076/2115] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:48.871942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.879038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1040/2116] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:48.880119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.885610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e154f-b4c7-4fdd-846a-56b84e84ca75 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.891272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e27bef-5900-4910-969e-ded3caf0265b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.891401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.891984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.892277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.892531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.893180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.894929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.896030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a7969-4122-4709-8873-c49b8a22bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.896625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a7969-4122-4709-8873-c49b8a22bc90 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.896904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a7969-4122-4709-8873-c49b8a22bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.897488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a7969-4122-4709-8873-c49b8a22bc90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.897804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a7969-4122-4709-8873-c49b8a22bc90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.899959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a7969-4122-4709-8873-c49b8a22bc90 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.917383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1077/2117] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:48.918392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:48.930906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351433fd-4d55-4d15-b9ba-1c0055aca1f0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:48.936536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:48.937337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:48.937712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:48.938087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] PATH_INFO: `/v3/projects/dfc3b58c039d4dd59467c0f10128573c/users/d29dabdeaf5a461b936a934b315d101d/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:48.940390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc3b58c039d4dd59467c0f10128573c, user_id=d29dabdeaf5a461b936a934b315d101d, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:48.943581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:48.972855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1078/2118] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:48 2022] PUT /identity/v3/projects/dfc3b58c039d4dd59467c0f10128573c/users/d29dabdeaf5a461b936a934b315d101d/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:48.974041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.051323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfcbb5-eb99-417c-b2ee-ffddb70b7703 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.065811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1041/2119] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:48 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:49.065811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b47eef-69b1-4899-bdcc-baa245de0567 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.066224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b47eef-69b1-4899-bdcc-baa245de0567 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.066710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b47eef-69b1-4899-bdcc-baa245de0567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.070534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.076331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.076614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.076890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] PATH_INFO: `/v3/projects/392829920ff54b09bdd2c75934153dfe/users/7ce2f809ba444e2e953abaf6a0b576c3/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.081447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] RBAC: Authorizing `identity:create_grant(project_id=392829920ff54b09bdd2c75934153dfe, user_id=7ce2f809ba444e2e953abaf6a0b576c3, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.084796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.100108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1079/2120] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:48 2022] PUT /identity/v3/projects/392829920ff54b09bdd2c75934153dfe/users/7ce2f809ba444e2e953abaf6a0b576c3/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:49.101070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.108557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-11b47eef-69b1-4899-bdcc-baa245de0567 None None] MFA Rules not processed for user `d29dabdeaf5a461b936a934b315d101d`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:49.109844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-11b47eef-69b1-4899-bdcc-baa245de0567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.153216 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:49.192953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746a62c-bf91-4ddc-b0c8-32df43de891e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.198017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.204165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.204165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.204165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.204165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.204165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.241816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1042/2121] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:49.244479 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.245874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.246932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.279126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] MFA Rules not processed for user `7ce2f809ba444e2e953abaf6a0b576c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:49.280333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.303783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1080/2122] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:49 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:49.304824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.306523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.360555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1043/2123] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:49.360555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.371130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da69f12-70f4-4c6d-84eb-57c9653d717b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.375991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.376590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.376867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.377139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.377785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.379497 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.381464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fcb95-97c6-4811-8120-27810d34ff21 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.385271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.385907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.386178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.389620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.394173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1044/2124] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:49.394173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.396277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.410723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f6884-98ac-4653-8c45-6016dca00d38 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.414016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.414449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.418420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.418420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.419241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.419401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.419401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.429529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1081/2125] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:49.430494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.444563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6619de-1df8-4bec-a929-c54aef31625c service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.450445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.452823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.453113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.453397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.454054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.455771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.470239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1082/2126] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:49.478535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.490226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176efabc-ae19-4c03-866b-227f0d78f2d9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.510609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed044b-ce76-4d2a-a8f5-a8a8ed6c9505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.510609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed044b-ce76-4d2a-a8f5-a8a8ed6c9505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.510609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed044b-ce76-4d2a-a8f5-a8a8ed6c9505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.510609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed044b-ce76-4d2a-a8f5-a8a8ed6c9505 admin admin] PATH_INFO: `/v3/projects/0a15ac32f8d9422c95795fb54745a10a/users/aaa7278dd81f44c7989b0abe55fb600c/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.526999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed044b-ce76-4d2a-a8f5-a8a8ed6c9505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a15ac32f8d9422c95795fb54745a10a, user_id=aaa7278dd81f44c7989b0abe55fb600c, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.534868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed044b-ce76-4d2a-a8f5-a8a8ed6c9505 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.547933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.549367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.561610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1045/2127] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:49.650569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:49.659255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:49.664162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec82a3-9d12-430c-8b0f-e132983efe41 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:49.686531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1083/2128] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:49 2022] PUT /identity/v3/projects/0a15ac32f8d9422c95795fb54745a10a/users/aaa7278dd81f44c7989b0abe55fb600c/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:49.695592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.698037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.698322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.782363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:49.783501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:49.783937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:49.784314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:49.786858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.796697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] MFA Rules not processed for user `aaa7278dd81f44c7989b0abe55fb600c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:49.798471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:49.825647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:49.944550 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:49.946197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:49.963867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1046/2129] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:49.976284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1084/2130] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:50.010601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:50.025127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345579de-5d0b-4993-9da8-a9aa313cb140 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:50.030092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:50.030739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:50.031019 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:50.031292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:50.046216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:50.048219 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:50.060333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1047/2131] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:50.193247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:50.437703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:50.439979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:50.571039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648f6ec-befd-43a4-b16c-d9ef278082e0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:50.577014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:50.577659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:50.577940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:50.578215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] PATH_INFO: `/v3/users/ddf07a7d8aed4b3faf058c66347cf315` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:50.579373 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf07a7d8aed4b3faf058c66347cf315)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:50.581351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:50.797303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf07a7d8aed4b3faf058c66347cf315'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:50.797961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf07a7d8aed4b3faf058c66347cf315'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:50.798589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf07a7d8aed4b3faf058c66347cf315'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:50.805651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf07a7d8aed4b3faf058c66347cf315'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:50.818324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf07a7d8aed4b3faf058c66347cf315'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:50.874165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:50.912016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1085/2132] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:50 2022] DELETE /identity/v3/users/ddf07a7d8aed4b3faf058c66347cf315 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:51.196582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:51.207851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2525-5364-4855-9719-8f7710b39807 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:51.252634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:51.260044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:51.266483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:51.279424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:51.280328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.280773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.281146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] PATH_INFO: `/v3/users/df80328a98704b4384eaac350c0e9b4d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.282577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df80328a98704b4384eaac350c0e9b4d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:51.289205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:51.345154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3351-f3c9-4c89-aba2-9b877bd59303 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:51.349689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:51.350316 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.350629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.350909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] PATH_INFO: `/v3/projects/d3e10ea750ae4455b7bc482b0db2ce6d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.352025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e10ea750ae4455b7bc482b0db2ce6d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:51.353970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:51.468460 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df80328a98704b4384eaac350c0e9b4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.469118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df80328a98704b4384eaac350c0e9b4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.469794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df80328a98704b4384eaac350c0e9b4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.471341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e10ea750ae4455b7bc482b0db2ce6d', 'request_id': 'req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.476829 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df80328a98704b4384eaac350c0e9b4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.488425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-19d26db3-4035-4ef3-9258-37ae06744a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df80328a98704b4384eaac350c0e9b4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.563452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e10ea750ae4455b7bc482b0db2ce6d', 'request_id': 'req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.573841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e10ea750ae4455b7bc482b0db2ce6d', 'request_id': 'req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.600187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] The token cache is being invalidate because project d3e10ea750ae4455b7bc482b0db2ce6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:51.600657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.601545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a53e36ba-37ef-4fe6-9fb8-c6492dcfbc84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:51.604098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1086/2133] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:51 2022] DELETE /identity/v3/projects/d3e10ea750ae4455b7bc482b0db2ce6d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:51.611810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ddf97-7e3e-491b-9eb5-a7df08769113 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.612671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ddf97-7e3e-491b-9eb5-a7df08769113 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.613028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ddf97-7e3e-491b-9eb5-a7df08769113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.639185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1048/2134] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:51 2022] DELETE /identity/v3/users/df80328a98704b4384eaac350c0e9b4d => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:51.640900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c61ddf97-7e3e-491b-9eb5-a7df08769113 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:51.641034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.641201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.641484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.642110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c61ddf97-7e3e-491b-9eb5-a7df08769113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:51.665602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:51.666070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:51.713875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1087/2135] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:51.723982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.724294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.724588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.744815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:51.745558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1049/2136] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:51.746052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:51.754319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1088/2137] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:51.755295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:51.756599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:51.758259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:51.767731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90c9a6-9d5f-4a33-9b72-75fb302e6c64 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:51.772445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:51.773029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.773306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.773581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.774219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:51.774520 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:51.776378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:51.789221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1089/2138] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:15:51.793684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:51.804438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac175f2-b1fc-406b-8240-b8e828129ef2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:51.806850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:51.811839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:51.812429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.812704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.812973 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.813600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:51.815617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:51.820084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b215cd-dac7-46b5-9c1a-b2fb3efb6842 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:51.824893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:51.825507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:51.825787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:51.826056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:51.836281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:51.837770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:51.851109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1050/2139] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:51.948703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1090/2140] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:51 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:51.953455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:51.965389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e72a67-f547-42d0-9523-1029a64e1a12 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:51.978418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:52.023814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.024542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.024741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.025013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.025640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.027865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.157222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1051/2141] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:51 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:52.161330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:52.175980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179edb98-750b-4553-b97b-aa748592209b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:52.182844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.184215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.184685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.185560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.186299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.188310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.203516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1091/2142] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:52.205729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:52.216827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e87c0-f142-4f2c-9318-4f2b043342fb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:52.221797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.222415 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.222709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.222981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.223753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.225494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.240725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1052/2143] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:52.243570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:52.252998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad378a-2212-4b58-9605-f91796325ba4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:52.258504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35415-4534-48f7-a69d-bf3c32e5b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.259276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35415-4534-48f7-a69d-bf3c32e5b5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.259661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35415-4534-48f7-a69d-bf3c32e5b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.260467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35415-4534-48f7-a69d-bf3c32e5b5e0 admin admin] PATH_INFO: `/v3/projects/f7e92f5c36664c8fb2495f17a913da5e/users/81059c0d734c4c1db4cd01e49cd3da7a/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.283390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35415-4534-48f7-a69d-bf3c32e5b5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e92f5c36664c8fb2495f17a913da5e, user_id=81059c0d734c4c1db4cd01e49cd3da7a, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.286134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35415-4534-48f7-a69d-bf3c32e5b5e0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.389939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:52.399903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1092/2144] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:52 2022] PUT /identity/v3/projects/f7e92f5c36664c8fb2495f17a913da5e/users/81059c0d734c4c1db4cd01e49cd3da7a/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:52.400881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:52.404617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.404993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.405338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.416132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d6fc2-0552-47c2-98db-2766707b1f96 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:52.463823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] MFA Rules not processed for user `81059c0d734c4c1db4cd01e49cd3da7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:52.463823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:52.486794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.487512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.487851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.488128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] PATH_INFO: `/v3/projects/3aa3291c0f334fc481f3e9027575d441` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.498725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa3291c0f334fc481f3e9027575d441)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.498725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.539019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1093/2145] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:52.543387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:52.556528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43997408-eebe-4a79-932c-a70c4d4668aa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:52.563451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.563451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.563451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.563451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.564191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.566440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.883076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa3291c0f334fc481f3e9027575d441', 'request_id': 'req-a5b4095a-7494-4aa0-acd6-62a2a445334e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:52.892659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa3291c0f334fc481f3e9027575d441', 'request_id': 'req-a5b4095a-7494-4aa0-acd6-62a2a445334e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:52.899863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa3291c0f334fc481f3e9027575d441', 'request_id': 'req-a5b4095a-7494-4aa0-acd6-62a2a445334e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:52.905024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1094/2146] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:52 2022] POST /identity/v3/projects => generated 381 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:52.906043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:52.972974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98f023-0d92-4147-aaac-4523a5c7f716 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:52.978665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:52.985743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:52.985743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:52.985743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:52.985743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:52.985743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:52.991026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:53.011725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] The token cache is being invalidate because project 3aa3291c0f334fc481f3e9027575d441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:53.014530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.014530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.014530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1053/2147] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:52 2022] DELETE /identity/v3/projects/3aa3291c0f334fc481f3e9027575d441 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:53.015069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.016339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:53.051409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:53.085404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1095/2148] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:52 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:53.086500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.088169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:53.088943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4095a-7494-4aa0-acd6-62a2a445334e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.097583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.098288 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.098586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.098860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] PATH_INFO: `/v3/users/4dc2e708ea964b11ab4cc4df205bfb91` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.100022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc2e708ea964b11ab4cc4df205bfb91)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.100799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa155-c71f-4b8b-bd31-10f8c54894a5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.102649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.106607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.107242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.107522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.107803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] PATH_INFO: `/v3/users/d29dabdeaf5a461b936a934b315d101d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.108942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29dabdeaf5a461b936a934b315d101d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.110876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.330832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc2e708ea964b11ab4cc4df205bfb91'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.331478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc2e708ea964b11ab4cc4df205bfb91'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.332076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc2e708ea964b11ab4cc4df205bfb91'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.339354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc2e708ea964b11ab4cc4df205bfb91'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.352490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc2e708ea964b11ab4cc4df205bfb91'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.372492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29dabdeaf5a461b936a934b315d101d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.373188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29dabdeaf5a461b936a934b315d101d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.373809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29dabdeaf5a461b936a934b315d101d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.381084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29dabdeaf5a461b936a934b315d101d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.394745 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-841d533a-55a7-4286-a686-215920f559e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29dabdeaf5a461b936a934b315d101d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:53.449129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1054/2149] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:53 2022] DELETE /identity/v3/users/4dc2e708ea964b11ab4cc4df205bfb91 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:53.451096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.453800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:53.536284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1096/2150] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:53 2022] DELETE /identity/v3/users/d29dabdeaf5a461b936a934b315d101d => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:53.537524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.537807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.538080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.564344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a1d8a-18c3-4da9-b1e5-c174b4f78b6e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.579844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:53.581343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:53.637818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:53.676635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.676635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.676635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.676635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.677632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.677632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.682436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1055/2151] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:53.687262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.690854 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:53.710440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb1a45-91dd-4193-90e4-2bb08298e237 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.711896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1097/2152] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:53.716692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.717420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.717791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.718174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] PATH_INFO: `/v3/projects/33111b2e1e9b41e0bb496ba85a00d452/users/801442e59df34ffc940a812908b652c5/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.727696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.729281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:53.736111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33111b2e1e9b41e0bb496ba85a00d452, user_id=801442e59df34ffc940a812908b652c5, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.739149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.791342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a989fc-f08e-49f0-bcc6-06d8ca4a73c3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.796025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.805894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.805894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.805894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.808228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.809665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.825726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1098/2153] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:53.870713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1056/2154] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:53 2022] PUT /identity/v3/projects/33111b2e1e9b41e0bb496ba85a00d452/users/801442e59df34ffc940a812908b652c5/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:53.874690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.882207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:53.885967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85b34-e71e-4c9e-a114-a28681cabb49 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.946628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ea760-a4bb-4710-97b1-90d7a96a100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.947252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea760-a4bb-4710-97b1-90d7a96a100c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.947549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea760-a4bb-4710-97b1-90d7a96a100c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.947820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ea760-a4bb-4710-97b1-90d7a96a100c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.948456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ea760-a4bb-4710-97b1-90d7a96a100c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.950699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ea760-a4bb-4710-97b1-90d7a96a100c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.960239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1099/2155] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:53.964575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:53.974662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8f15a-5fec-48ba-85d2-8c53cefea1a4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:53.979268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:53.979902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:53.980217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:53.980483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] PATH_INFO: `/v3/projects/33111b2e1e9b41e0bb496ba85a00d452/users/801442e59df34ffc940a812908b652c5/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:53.982352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=33111b2e1e9b41e0bb496ba85a00d452, user_id=801442e59df34ffc940a812908b652c5, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:53.985022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:53.995034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1057/2156] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:53 2022] PUT /identity/v3/projects/33111b2e1e9b41e0bb496ba85a00d452/users/801442e59df34ffc940a812908b652c5/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:54.006966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.007360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.007640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.036201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] MFA Rules not processed for user `801442e59df34ffc940a812908b652c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:54.037427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.109296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1100/2157] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:54.120557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.123948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.193731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.193731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.200183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597477b1-82e5-45cd-8e77-01d264792ada admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.218938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.220064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.220633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.220916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.244770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.246970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.263441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1058/2158] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:15:54.299388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c630-2d72-4786-a07f-283e7031a1b6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.304910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.305543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.306739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.307017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] PATH_INFO: `/v3/projects/dfc3b58c039d4dd59467c0f10128573c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.309089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc3b58c039d4dd59467c0f10128573c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.311185 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.373655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.382484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88a45-e5a0-41bb-a314-0d4a8cfc8c8d service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.386768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.387378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.387656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.387934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.389790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.425735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:54.444021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc3b58c039d4dd59467c0f10128573c', 'request_id': 'req-77f8c218-fd89-43d3-af96-7438486c4408'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.459220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.460654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cafe3d-0b4b-48e0-b591-b2aba7e28082 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.470392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1059/2159] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:54.531718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc3b58c039d4dd59467c0f10128573c', 'request_id': 'req-77f8c218-fd89-43d3-af96-7438486c4408'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.539325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc3b58c039d4dd59467c0f10128573c', 'request_id': 'req-77f8c218-fd89-43d3-af96-7438486c4408'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.551291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] The token cache is being invalidate because project dfc3b58c039d4dd59467c0f10128573c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:54.551705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.552621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.554380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1101/2160] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:54 2022] DELETE /identity/v3/projects/dfc3b58c039d4dd59467c0f10128573c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:54.560142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c7a7-5e9b-4509-a0f9-f82aeeec499d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.562372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c7a7-5e9b-4509-a0f9-f82aeeec499d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.562372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57c7a7-5e9b-4509-a0f9-f82aeeec499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.581471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-de57c7a7-5e9b-4509-a0f9-f82aeeec499d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:54.582707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-de57c7a7-5e9b-4509-a0f9-f82aeeec499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.634990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:54.646332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.647889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.651136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1060/2161] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:54.657516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.657802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.658078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.661905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8c218-fd89-43d3-af96-7438486c4408 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.668111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.668777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.669065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.669498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] PATH_INFO: `/v3/projects/f219be7ffd74405fa8041e17b4e43b0b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.675743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f219be7ffd74405fa8041e17b4e43b0b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.678376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.696992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:54.699520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.720015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1061/2162] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:54.724480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.735722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34793a-010c-4280-bfe9-59ba459e7315 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.741106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.743446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.743446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.743446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.743446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:54.743446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.745216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.773580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1062/2163] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:15:54.779091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.788027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f219be7ffd74405fa8041e17b4e43b0b', 'request_id': 'req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.793257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d665ecf-462b-47d8-b754-8aacb726e3f2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.861903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.862504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.862806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.863076 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.863699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.865616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.902638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f219be7ffd74405fa8041e17b4e43b0b', 'request_id': 'req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.909410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f219be7ffd74405fa8041e17b4e43b0b', 'request_id': 'req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.920533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] The token cache is being invalidate because project f219be7ffd74405fa8041e17b4e43b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:54.920820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.921456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:54.922997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1102/2164] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:54 2022] DELETE /identity/v3/projects/f219be7ffd74405fa8041e17b4e43b0b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:54.926934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.928217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:54.941865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c83a8e-a4f4-468b-a552-019bb1c24c1c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:54.947136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:54.947793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:54.948075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:54.948416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] PATH_INFO: `/v3/users/7ce2f809ba444e2e953abaf6a0b576c3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:54.949625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce2f809ba444e2e953abaf6a0b576c3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:54.951605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:54.982266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1063/2165] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:54 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:54.986462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:54.988086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:55.051382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef909ae-8fb7-4afe-b4aa-582a1eb0bf16 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.056382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.061079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.061079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.061079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.061079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.061079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.167203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1064/2166] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:54 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:55.172002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.176813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce2f809ba444e2e953abaf6a0b576c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.177531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce2f809ba444e2e953abaf6a0b576c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.178201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce2f809ba444e2e953abaf6a0b576c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.184728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bc211-9745-45a1-8a04-8fbbfaf84565 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.188066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce2f809ba444e2e953abaf6a0b576c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.189695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.190293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.190593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.190866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.191635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.193297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.205024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ac3600e-d9a0-439e-a7f8-f5d8e99081cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce2f809ba444e2e953abaf6a0b576c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.207813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1065/2167] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:55.208683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.217070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:55.237166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77fe4-c76f-4aeb-a0d3-70c96bffb9c3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.242277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.245915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.245915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.245915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.245915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.245915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.258408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1066/2168] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:55.263407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.273093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271eb88-4302-4585-9799-14b3f0c28eb5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.278606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.279236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.279516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.279784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] PATH_INFO: `/v3/projects/4d8bcc347dc54dd9910820347aaf7bda/users/ac331b9957094f81ae41da9f9cc67e65/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.294915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8bcc347dc54dd9910820347aaf7bda, user_id=ac331b9957094f81ae41da9f9cc67e65, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.297782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.300836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1103/2169] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:15:54 2022] DELETE /identity/v3/users/7ce2f809ba444e2e953abaf6a0b576c3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:55.374561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1067/2170] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:55 2022] PUT /identity/v3/projects/4d8bcc347dc54dd9910820347aaf7bda/users/ac331b9957094f81ae41da9f9cc67e65/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:55.386099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.386380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.386687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.408327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=75,keystone:DELETE=22,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:55.416041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] MFA Rules not processed for user `ac331b9957094f81ae41da9f9cc67e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:55.417207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:55.499065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1104/2171] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:55.503661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.508416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:55.576063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088a628-9847-41a5-9a3b-d39691c49b87 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.581102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.581708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.581985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.582258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.582962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.584918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.695482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1068/2172] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:55.700233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.711025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f52f51-216b-4253-9acd-e0b7b756ce93 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.747560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.749407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:55.779717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.780638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.781094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.781411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.782133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.786375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.839068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a03a0-787d-4dbc-9c68-ddbf7d273a56 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.846044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.846894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.847274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.847655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] PATH_INFO: `/v3/projects/392829920ff54b09bdd2c75934153dfe` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.848881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=392829920ff54b09bdd2c75934153dfe)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.850979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.885088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1105/2173] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:55 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:55.889986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.902630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c3e7b-94ea-42e8-bcb7-6d3a9a6466ed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.908267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:55.909198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:55.910144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:55.910144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:55.910644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:55.916610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:55.924079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1106/2174] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:55.928075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:55.951461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56382ea-d880-4f7e-b733-796b71cb8707 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:55.959874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392829920ff54b09bdd2c75934153dfe', 'request_id': 'req-12454eb4-8e03-422f-836f-f2696ac505cf'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.967405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392829920ff54b09bdd2c75934153dfe', 'request_id': 'req-12454eb4-8e03-422f-836f-f2696ac505cf'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:55.973786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392829920ff54b09bdd2c75934153dfe', 'request_id': 'req-12454eb4-8e03-422f-836f-f2696ac505cf'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:56.021303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.021908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.022186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.022455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.023120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.030717 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.034537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1107/2175] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:56.038616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.048223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b9715-0e1e-476e-931c-6226ddc59766 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.052963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4335abe-71b4-4206-99bf-47325b064f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.057726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4335abe-71b4-4206-99bf-47325b064f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.057726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4335abe-71b4-4206-99bf-47325b064f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.057726 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4335abe-71b4-4206-99bf-47325b064f08 admin admin] PATH_INFO: `/v3/projects/819929ac35b649f0980a0935e204ae2e/users/ea8952533e924ae086059dc74c52c673/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.066645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:56.069678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4335abe-71b4-4206-99bf-47325b064f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819929ac35b649f0980a0935e204ae2e, user_id=ea8952533e924ae086059dc74c52c673, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.072329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4335abe-71b4-4206-99bf-47325b064f08 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.082209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] The token cache is being invalidate because project 392829920ff54b09bdd2c75934153dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:15:56.082498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:56.088203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-12454eb4-8e03-422f-836f-f2696ac505cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:15:56.088203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1069/2176] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:15:55 2022] DELETE /identity/v3/projects/392829920ff54b09bdd2c75934153dfe => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:56.089502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9861053-a93f-468d-b6bd-5291ea40b341 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.089786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9861053-a93f-468d-b6bd-5291ea40b341 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.090086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9861053-a93f-468d-b6bd-5291ea40b341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.109778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a9861053-a93f-468d-b6bd-5291ea40b341 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:56.111082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a9861053-a93f-468d-b6bd-5291ea40b341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:56.118986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1070/2177] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:56.126913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.127194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.127465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.146643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:56.147816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:56.155764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1071/2178] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:56.159590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.165530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:56.169310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1108/2179] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:56 2022] PUT /identity/v3/projects/819929ac35b649f0980a0935e204ae2e/users/ea8952533e924ae086059dc74c52c673/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:56.172528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310260e9-5cc3-40fd-a7cf-8f1b08d52924 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.183774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b15a3-3b77-4697-b133-893e310e217f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.184058 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b15a3-3b77-4697-b133-893e310e217f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.184332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b15a3-3b77-4697-b133-893e310e217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.213665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-901b15a3-3b77-4697-b133-893e310e217f None None] MFA Rules not processed for user `ea8952533e924ae086059dc74c52c673`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:56.215082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-901b15a3-3b77-4697-b133-893e310e217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:56.235449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.236263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.236760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.237176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.237832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:15:56.238132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.239840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.252465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1072/2180] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:15:56.256593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.267819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009664f0-5c53-4735-b161-7e005bd90e2b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.272582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.273175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.273446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.273712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.274344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.276519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.299341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1109/2181] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:56.303684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b15a3-3b77-4697-b133-893e310e217f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.305088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-901b15a3-3b77-4697-b133-893e310e217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:56.318577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b15a3-3b77-4697-b133-893e310e217f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.323145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.323792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.327883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.327883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.327883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.327883 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.380609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1073/2182] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:56.384410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.446209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c043d6-d106-4845-a59e-26cba9614dce admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.450985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.451580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.451851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.452118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.452744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.454674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.472670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1110/2183] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:56.476575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.540548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a25a0-279f-487f-98be-b0e1e37bcdad admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.546340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.546982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.547287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.547566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.548204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.550108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.570722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1074/2184] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:15:56.576131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.587585 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc2444-7156-4503-85aa-94196561db04 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.592325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.596957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.596957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.596957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.596957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.596957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.607192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1075/2185] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:56.610918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.621976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6b00-91de-4b72-9c25-f07103697607 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.626745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.627384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.627676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.627948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.628592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.630185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.641211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1076/2186] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:56.645850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.652328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1111/2187] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:15:56.656311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7ed8-16c9-477f-bde1-3c4f794895c9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.656540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.664969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3acaca-2339-4d39-8ead-beb5892b611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.666248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3acaca-2339-4d39-8ead-beb5892b611a admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.666472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3acaca-2339-4d39-8ead-beb5892b611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.667031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3acaca-2339-4d39-8ead-beb5892b611a admin admin] PATH_INFO: `/v3/projects/aade1725a943407b9baf6f3eb1fdb46f/users/3ae692f22824423bbfa52429ccf5aa9c/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.668595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d24357-c712-422a-b6f8-dc3b3c6a99e7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.674377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.675083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.678337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.678337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.678337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.678337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.687175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3acaca-2339-4d39-8ead-beb5892b611a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aade1725a943407b9baf6f3eb1fdb46f, user_id=3ae692f22824423bbfa52429ccf5aa9c, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.690099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3acaca-2339-4d39-8ead-beb5892b611a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.694624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1112/2188] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:15:56.695746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.705556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7dab0-2630-4076-b8b2-dd0c72c09089 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.710467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:56.711271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.711599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.711875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] PATH_INFO: `/v3/projects/88a3ad13e32b4ab9af05d7fd559b8ddb/users/1dcaf1e5eb804e8e8f9d5147f89e72f2/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.730527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a3ad13e32b4ab9af05d7fd559b8ddb, user_id=1dcaf1e5eb804e8e8f9d5147f89e72f2, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:56.733146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:56.811663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1113/2189] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:56 2022] PUT /identity/v3/projects/88a3ad13e32b4ab9af05d7fd559b8ddb/users/1dcaf1e5eb804e8e8f9d5147f89e72f2/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:56.811978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1077/2190] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:56 2022] PUT /identity/v3/projects/aade1725a943407b9baf6f3eb1fdb46f/users/3ae692f22824423bbfa52429ccf5aa9c/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:15:56.816626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.825278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:56.825566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:56.825848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:56.831006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:56.857381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] MFA Rules not processed for user `3ae692f22824423bbfa52429ccf5aa9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:56.858726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:56.868107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:56.935793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50933-ea05-4268-ae79-7b5bfd5a8413 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:56.958834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1078/2191] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:15:56.971805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:56.973377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:57.045257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:57.045885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:57.055344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:57.055344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:57.055344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:57.055344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:57.066746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1114/2192] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:15:57.070504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:57.080412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d1ac-e120-4925-b561-17f20bda12f3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:57.082682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce115344-0819-43c9-a239-a5873171331b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:57.084805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:57.085526 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:57.085889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:57.086853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:57.089005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8d483-0d79-4758-aba5-364e8bb4e3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:57.089846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8d483-0d79-4758-aba5-364e8bb4e3df admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:57.090228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8d483-0d79-4758-aba5-364e8bb4e3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:57.090667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8d483-0d79-4758-aba5-364e8bb4e3df admin admin] PATH_INFO: `/v3/projects/88a3ad13e32b4ab9af05d7fd559b8ddb/users/1dcaf1e5eb804e8e8f9d5147f89e72f2/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:57.092775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8d483-0d79-4758-aba5-364e8bb4e3df admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a3ad13e32b4ab9af05d7fd559b8ddb, user_id=1dcaf1e5eb804e8e8f9d5147f89e72f2, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:57.096080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8d483-0d79-4758-aba5-364e8bb4e3df admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:57.106672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:57.110337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:57.112874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1115/2193] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:15:57 2022] PUT /identity/v3/projects/88a3ad13e32b4ab9af05d7fd559b8ddb/users/1dcaf1e5eb804e8e8f9d5147f89e72f2/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:15:57.125578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:57.126320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:57.126898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:57.132601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1079/2194] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:15:57.190051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] MFA Rules not processed for user `1dcaf1e5eb804e8e8f9d5147f89e72f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:15:57.190685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:57.295008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1116/2195] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:15:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:15:57.676518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:15:57.678164 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:57.749566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921967-e6f8-4605-8e4a-a53266862f9c service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:15:57.800352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:15:57.800959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:15:57.801238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:15:57.801511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:15:57.803307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:15:57.825474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:15:57.864367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:15:57.865872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:15:57.877110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1080/2196] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:15:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:02.120045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:02.130051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb337b3-7182-46c3-ba69-05ebf152b235 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:02.133699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468d63a-6a42-4e44-9125-58604a5ef0d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:02.134302 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468d63a-6a42-4e44-9125-58604a5ef0d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:02.134612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468d63a-6a42-4e44-9125-58604a5ef0d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:02.134874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468d63a-6a42-4e44-9125-58604a5ef0d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:02.164548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:02.195012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468d63a-6a42-4e44-9125-58604a5ef0d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:02.196465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468d63a-6a42-4e44-9125-58604a5ef0d1 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:02.206464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1117/2197] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:02.564215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:02.572789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:02.642056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefbc44-ab96-44c6-a2c0-436068b93c1a service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:02.646692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:02.647311 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:02.647593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:02.647871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] PATH_INFO: `/v3/users/81059c0d734c4c1db4cd01e49cd3da7a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:02.648999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=81059c0d734c4c1db4cd01e49cd3da7a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:02.651235 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:02.908729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81059c0d734c4c1db4cd01e49cd3da7a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:02.909393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81059c0d734c4c1db4cd01e49cd3da7a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:02.910004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81059c0d734c4c1db4cd01e49cd3da7a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:02.918194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81059c0d734c4c1db4cd01e49cd3da7a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:02.934342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81059c0d734c4c1db4cd01e49cd3da7a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:03.045074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1081/2198] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:02 2022] DELETE /identity/v3/users/81059c0d734c4c1db4cd01e49cd3da7a => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:03.050048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:03.057031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:03.057031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:03.070081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:03.071342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:03.133556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1118/2199] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:03.144058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:03.145481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:03.202464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:03.214855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125620-ff00-4630-89c5-19e6476800cf admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:03.218681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:03.219373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:03.219707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:03.220040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:03.230315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:03.231907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:03.242972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1082/2200] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:03.656715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:03.658372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:03.671135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70addb9a-12e0-4c7b-bf04-d7cd79b8615a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:03.676715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:03.677341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:03.677619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:03.678013 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] PATH_INFO: `/v3/projects/f7e92f5c36664c8fb2495f17a913da5e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:03.679175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e92f5c36664c8fb2495f17a913da5e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:03.681285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:03.835297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e92f5c36664c8fb2495f17a913da5e', 'request_id': 'req-7057be0e-7262-4204-a78a-99ac41cbceb8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:03.939918 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e92f5c36664c8fb2495f17a913da5e', 'request_id': 'req-7057be0e-7262-4204-a78a-99ac41cbceb8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:03.952679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e92f5c36664c8fb2495f17a913da5e', 'request_id': 'req-7057be0e-7262-4204-a78a-99ac41cbceb8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:03.969429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] The token cache is being invalidate because project f7e92f5c36664c8fb2495f17a913da5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:03.969792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:03.970618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:03.980886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1119/2201] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:03 2022] DELETE /identity/v3/projects/f7e92f5c36664c8fb2495f17a913da5e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:03.989938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:04.002313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5f3d-84e0-441b-b692-f268d5b94416 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:04.065597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:04.066326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:04.066603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:04.066893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] PATH_INFO: `/v3/users/801442e59df34ffc940a812908b652c5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:04.068045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=801442e59df34ffc940a812908b652c5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:04.070123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:04.146447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:04.148167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:04.218106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be0e-7262-4204-a78a-99ac41cbceb8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:04.221956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:04.222613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:04.222896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:04.223175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:04.225014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:04.230741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:04.291996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:04.293576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:04.310158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1120/2202] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:04.315275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801442e59df34ffc940a812908b652c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:04.316237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801442e59df34ffc940a812908b652c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:04.316986 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801442e59df34ffc940a812908b652c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:04.326215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801442e59df34ffc940a812908b652c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:04.340158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f43cf3d7-c425-4ce9-ab7e-89458f7b2ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801442e59df34ffc940a812908b652c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:04.406403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1083/2203] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:03 2022] DELETE /identity/v3/users/801442e59df34ffc940a812908b652c5 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:04.856858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:04.858294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:04.875405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f920e-8fe7-425f-8a78-9a4906f9063e service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:04.935498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:04.936182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:04.936460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:04.936757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] PATH_INFO: `/v3/projects/33111b2e1e9b41e0bb496ba85a00d452` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:04.937901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33111b2e1e9b41e0bb496ba85a00d452)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:04.939864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.045370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33111b2e1e9b41e0bb496ba85a00d452', 'request_id': 'req-682444fa-fb24-437a-a25b-e7b2ceeeeb22'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:05.139176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33111b2e1e9b41e0bb496ba85a00d452', 'request_id': 'req-682444fa-fb24-437a-a25b-e7b2ceeeeb22'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:05.146002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33111b2e1e9b41e0bb496ba85a00d452', 'request_id': 'req-682444fa-fb24-437a-a25b-e7b2ceeeeb22'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:05.157731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] The token cache is being invalidate because project 33111b2e1e9b41e0bb496ba85a00d452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:05.158314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:05.158721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-682444fa-fb24-437a-a25b-e7b2ceeeeb22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:05.160227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1121/2204] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:04 2022] DELETE /identity/v3/projects/33111b2e1e9b41e0bb496ba85a00d452 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:05.165156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.165586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.165709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.187712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:05.188942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:05.251813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1084/2205] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:05.259533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.259846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.260128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.280870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:05.282088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:05.289985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1122/2206] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:05.296568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.302652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:05.306374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d01d9-1a27-4f63-bb60-d485b8264183 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:05.311036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:05.311745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.312024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.312566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.313221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:05.313529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:05.315207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.327653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1085/2207] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:16:05.332190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.342310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2f285-84f5-4ea8-a2d0-3b74ce2f41d5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:05.347309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:05.347906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.348186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.348458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.349123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:05.351039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.435807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1123/2208] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:05 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:05.439951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.451608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8287d-9b6a-4a03-bb7e-8b165b33081f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:05.508231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:05.508848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.509129 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.509408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.510050 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:05.513258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.598791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1086/2209] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:05 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:05.607118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.613536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:05.622808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ee4d-d1f5-4ce5-a1e9-e358aef8a5f9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:05.627614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:05.628217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.628494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.628766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.629408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:05.631017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.642569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1124/2210] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:05.649471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.672385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cee9db-19c3-48fc-9671-6c12e1c70cc3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:05.673326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2678a20-3084-48fd-91dc-543a1ed457de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:05.674097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2678a20-3084-48fd-91dc-543a1ed457de admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.674419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2678a20-3084-48fd-91dc-543a1ed457de admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.674798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2678a20-3084-48fd-91dc-543a1ed457de admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.675721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2678a20-3084-48fd-91dc-543a1ed457de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:05.677508 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2678a20-3084-48fd-91dc-543a1ed457de admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.704816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1087/2211] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:05.714179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.725410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68d359-4756-4542-b3fa-85af46fc542e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:05.731525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:05.732213 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.736344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.736344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] PATH_INFO: `/v3/projects/acc8d05179964efb9c80f5d33c74cd3f/users/3acd0aed6f9946c5b895341708347ec6/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.752502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc8d05179964efb9c80f5d33c74cd3f, user_id=3acd0aed6f9946c5b895341708347ec6, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:05.767650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:05.840060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1125/2212] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:05 2022] PUT /identity/v3/projects/acc8d05179964efb9c80f5d33c74cd3f/users/3acd0aed6f9946c5b895341708347ec6/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:05.846926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8918-f34b-41e0-9d86-27f166452f13 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:05.847099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8918-f34b-41e0-9d86-27f166452f13 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:05.847283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8918-f34b-41e0-9d86-27f166452f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:05.866956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b66a8918-f34b-41e0-9d86-27f166452f13 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:05.868408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b66a8918-f34b-41e0-9d86-27f166452f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:05.931241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1088/2213] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:05.941814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:05.943571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:06.005093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c100845-13ac-40a7-9296-13f63021337d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:06.009234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:06.009996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:06.010231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:06.010503 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:06.020819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:06.022237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:06.033113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1126/2214] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:07.265777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.266248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.266326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.303225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] MFA Rules not processed for user `3acd0aed6f9946c5b895341708347ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:07.305126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:07.367971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:07.372247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1089/2215] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:07.377189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:07.387156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4ae72-7022-4edc-831c-e5ca27e79f91 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:07.391796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:07.392402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.392682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.392958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.393597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:07.395538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:07.487293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1127/2216] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:07 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:07.488651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:07.499338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf146-db0f-4a21-b048-7d95e8a87799 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:07.556714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:07.669431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.669431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.669431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.669431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:07.669431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:07.711598 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1090/2217] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:07 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:07.715365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:07.725519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2d8a-3fbf-43fb-8646-d0dd6fc9a421 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:07.729995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:07.731104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.731381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.731645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.732342 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:07.733909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:07.747069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1128/2218] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:07.749448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:07.759562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0535e-6aa5-4db0-bd5c-62259ae71687 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:07.769827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:07.770921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.771209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.771634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] PATH_INFO: `/v3/projects/f527746e49b24de4aa96b69822a477ff/users/15f5587fe7224113b285d38e6fd37416/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.789105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f527746e49b24de4aa96b69822a477ff, user_id=15f5587fe7224113b285d38e6fd37416, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:07.792928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:07.860418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1091/2219] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:07 2022] PUT /identity/v3/projects/f527746e49b24de4aa96b69822a477ff/users/15f5587fe7224113b285d38e6fd37416/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:07.864552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:07.875266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127908-aacb-4a46-b39d-f9f5301be9fd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:07.933332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb89f84-ef88-4743-95f4-4769c97e8fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:07.933967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb89f84-ef88-4743-95f4-4769c97e8fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.934244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb89f84-ef88-4743-95f4-4769c97e8fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.934517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb89f84-ef88-4743-95f4-4769c97e8fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.935189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb89f84-ef88-4743-95f4-4769c97e8fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:07.936780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb89f84-ef88-4743-95f4-4769c97e8fe6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:07.947182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1129/2220] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:07.951523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:07.962140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644b9d7-cbc8-4d64-b3c1-5d0011d71f12 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:07.967196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:07.967840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:07.968146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:07.968426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] PATH_INFO: `/v3/projects/f527746e49b24de4aa96b69822a477ff/users/15f5587fe7224113b285d38e6fd37416/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:07.970314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f527746e49b24de4aa96b69822a477ff, user_id=15f5587fe7224113b285d38e6fd37416, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:07.972987 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:07.985063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1092/2221] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:07 2022] PUT /identity/v3/projects/f527746e49b24de4aa96b69822a477ff/users/15f5587fe7224113b285d38e6fd37416/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:09.411526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:09.412001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:09.412101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:09.444017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] MFA Rules not processed for user `15f5587fe7224113b285d38e6fd37416`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:09.445594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:09.476690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:09.544640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1130/2222] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:09.594543 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:09.596381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:09.664681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6ee4-bc91-4eff-a8d3-01cad4df1083 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:09.669046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555139f6-b7ac-4fc4-bd06-e9539cfc7923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:09.669650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555139f6-b7ac-4fc4-bd06-e9539cfc7923 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:09.669928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555139f6-b7ac-4fc4-bd06-e9539cfc7923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:09.670204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555139f6-b7ac-4fc4-bd06-e9539cfc7923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:09.733534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555139f6-b7ac-4fc4-bd06-e9539cfc7923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:09.735029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555139f6-b7ac-4fc4-bd06-e9539cfc7923 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:09.744980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1093/2223] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:09.790391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:09.792156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:09.866414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac383d34-bb67-4c7b-a6ba-7128ed62d316 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:09.869706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:09.870354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:09.870664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:09.870938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] PATH_INFO: `/v3/users/a99a78a868c54c44be83c061b12e759d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:09.872321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99a78a868c54c44be83c061b12e759d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:09.874271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:10.073921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99a78a868c54c44be83c061b12e759d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:10.075018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99a78a868c54c44be83c061b12e759d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:10.075619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99a78a868c54c44be83c061b12e759d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:10.082937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99a78a868c54c44be83c061b12e759d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:10.096280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99a78a868c54c44be83c061b12e759d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:10.186682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1131/2224] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:09 2022] DELETE /identity/v3/users/a99a78a868c54c44be83c061b12e759d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:10.191227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:10.195279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:10.195279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:10.205655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:10.213790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:10.215156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:10.280067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1094/2225] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:10.290036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:10.292476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:10.344721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:10.362065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e9839-22a6-44d3-937f-e5f1b0750762 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:10.366882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:10.367481 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:10.367760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:10.368032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:10.380205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:10.394982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:10.418280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1132/2226] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:10.950648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:10.952389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:10.966707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717be4e-89f4-4f22-a097-b8c142efdc24 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:10.972573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:10.973254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:10.973481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:10.974212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] PATH_INFO: `/v3/projects/0b5b2b40cde24cd8976f795fab6292b0` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:10.975428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5b2b40cde24cd8976f795fab6292b0)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:10.977461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:11.113000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5b2b40cde24cd8976f795fab6292b0', 'request_id': 'req-6331d2e5-165b-4bf6-b541-72260a2a0714'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.120774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5b2b40cde24cd8976f795fab6292b0', 'request_id': 'req-6331d2e5-165b-4bf6-b541-72260a2a0714'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.128697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5b2b40cde24cd8976f795fab6292b0', 'request_id': 'req-6331d2e5-165b-4bf6-b541-72260a2a0714'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.226576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] The token cache is being invalidate because project 0b5b2b40cde24cd8976f795fab6292b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:11.227516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.228212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.229768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1095/2227] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:10 2022] DELETE /identity/v3/projects/0b5b2b40cde24cd8976f795fab6292b0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:11.239454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:11.260705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f41ef-1d5f-4bab-91dd-90611260387a service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:11.344991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:11.345550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:11.345829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:11.346105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] PATH_INFO: `/v3/users/6454972e80134e7785ce97417fd9f03b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:11.347254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6454972e80134e7785ce97417fd9f03b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:11.349188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:11.530376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:11.531882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:11.533853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6454972e80134e7785ce97417fd9f03b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.534643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6454972e80134e7785ce97417fd9f03b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.535325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6454972e80134e7785ce97417fd9f03b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.547804 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6454972e80134e7785ce97417fd9f03b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.563420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6454972e80134e7785ce97417fd9f03b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:11.604540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d2e5-165b-4bf6-b541-72260a2a0714 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:11.609825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:11.610440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:11.610793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:11.611117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:11.613028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:11.662557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:11.668840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1133/2228] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:11 2022] DELETE /identity/v3/users/6454972e80134e7785ce97417fd9f03b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:11.685424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:11.687468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:11.699051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1096/2229] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:12.165799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:12.165799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:12.282359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da1fd0-988b-478c-a439-0b241d89f019 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:12.287896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:12.288781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.288781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.289066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] PATH_INFO: `/v3/projects/deada32cf59e47208dcd1c55ab47af29` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.290191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deada32cf59e47208dcd1c55ab47af29)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:12.292116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:12.294236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:12.432604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:12.434125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:12.447862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02525c89-090d-43d5-8108-2eb5566d242d service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:12.452782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:12.453404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.457238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.457238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] PATH_INFO: `/v3/users/ac331b9957094f81ae41da9f9cc67e65` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.457238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac331b9957094f81ae41da9f9cc67e65)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:12.457806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:12.468989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deada32cf59e47208dcd1c55ab47af29', 'request_id': 'req-448338da-a334-40cb-9aea-241cb722d612'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.547841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deada32cf59e47208dcd1c55ab47af29', 'request_id': 'req-448338da-a334-40cb-9aea-241cb722d612'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.559331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deada32cf59e47208dcd1c55ab47af29', 'request_id': 'req-448338da-a334-40cb-9aea-241cb722d612'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.571780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] The token cache is being invalidate because project deada32cf59e47208dcd1c55ab47af29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:12.572070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.572713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-448338da-a334-40cb-9aea-241cb722d612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.574357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1134/2230] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:12 2022] DELETE /identity/v3/projects/deada32cf59e47208dcd1c55ab47af29 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:12.579494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaf58a-2677-4a06-b31a-efb015f510f1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.583403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaf58a-2677-4a06-b31a-efb015f510f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.583403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaf58a-2677-4a06-b31a-efb015f510f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.601092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4dfaf58a-2677-4a06-b31a-efb015f510f1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:12.602304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4dfaf58a-2677-4a06-b31a-efb015f510f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:12.666715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1135/2231] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:12.675052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.675341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.675618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.692712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac331b9957094f81ae41da9f9cc67e65'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.693396 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac331b9957094f81ae41da9f9cc67e65'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.694176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac331b9957094f81ae41da9f9cc67e65'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.695364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:12.696588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:12.701293 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac331b9957094f81ae41da9f9cc67e65'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.704701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1136/2232] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:12.708959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:12.713748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6604a98-8194-4c1d-aa9a-6b5c6332bf2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac331b9957094f81ae41da9f9cc67e65'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:12.718582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf953128-aaf2-404f-8833-a6697ec9f872 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:12.723298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:12.723878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.724155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.724429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.725068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:12.725395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:12.727021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:12.738928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1137/2233] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:16:12.743066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:12.752612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565a029-02a0-4009-9b11-f443d6ff8ff2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:12.757590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:12.758231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.758511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.758821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.759470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:12.761372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:12.873384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1097/2234] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:12 2022] DELETE /identity/v3/users/ac331b9957094f81ae41da9f9cc67e65 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:12.878328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1138/2235] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:12 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:12.878777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.879198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.879538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.883135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:12.907877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:12.909106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:12.967186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:12.981214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76a46-ac4c-45cb-a230-2b494f654279 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:12.989573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1098/2236] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:12.992616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:12.993255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:12.993542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:12.993819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:12.994476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:12.996526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.000055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.001493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:13.061264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d0854-13d9-4d74-9ecc-c717443f06ed None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.065032 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.065632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.065904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.066171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.073145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:13.077185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.081120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.091641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1099/2237] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:13.108777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1139/2238] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:12 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:13.109820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.119637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3c36-d619-4d54-9ca9-77c4dd2deb6b service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.125175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.125954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.126240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.126501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.127181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.128836 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.140114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1100/2239] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:13.145370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.155953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc93f4-f167-443d-b7b4-b30d1afb8466 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.160878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb93ab-da52-410b-bb99-473d217b6233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.161538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb93ab-da52-410b-bb99-473d217b6233 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.161853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb93ab-da52-410b-bb99-473d217b6233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.162130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb93ab-da52-410b-bb99-473d217b6233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.162809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb93ab-da52-410b-bb99-473d217b6233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.164413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb93ab-da52-410b-bb99-473d217b6233 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.178587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1140/2240] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:13.179399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.190102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dd843-c892-43b6-923f-c767cbcb07bb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.195109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.195766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.196048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.196433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] PATH_INFO: `/v3/projects/0de3278f10da45dfb6f53ef75e922c84/users/bd31c08d6e034ea3a3534ffe6561d246/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.212268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de3278f10da45dfb6f53ef75e922c84, user_id=bd31c08d6e034ea3a3534ffe6561d246, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.215118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.311855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1101/2241] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:13 2022] PUT /identity/v3/projects/0de3278f10da45dfb6f53ef75e922c84/users/bd31c08d6e034ea3a3534ffe6561d246/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:13.322565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.323041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.323699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.381490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] MFA Rules not processed for user `bd31c08d6e034ea3a3534ffe6561d246`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:13.383033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:13.475652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1141/2242] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:13.481324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.549793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac170-9e3e-42d6-9a6b-0724d9f8f6f9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.553968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.555472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.555622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:13.558339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.558657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.558936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.559611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.561710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.576084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f61f7-a5f1-424d-869d-97f128a7b900 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.584838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.585313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.585629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.585953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] PATH_INFO: `/v3/projects/4d8bcc347dc54dd9910820347aaf7bda` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.587069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8bcc347dc54dd9910820347aaf7bda)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.588981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.660198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1102/2243] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:13 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:13.661162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.719897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8bcc347dc54dd9910820347aaf7bda', 'request_id': 'req-ad899a95-a255-4f66-b35b-40f461951e61'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:13.734459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b09c27-ca08-4b74-aac9-268e654404b8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.792315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.793402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.793672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.793941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.794763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.796716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.807749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8bcc347dc54dd9910820347aaf7bda', 'request_id': 'req-ad899a95-a255-4f66-b35b-40f461951e61'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:13.815421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8bcc347dc54dd9910820347aaf7bda', 'request_id': 'req-ad899a95-a255-4f66-b35b-40f461951e61'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:13.825928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] The token cache is being invalidate because project 4d8bcc347dc54dd9910820347aaf7bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:13.826218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:13.826888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:13.828390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1142/2244] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:13 2022] DELETE /identity/v3/projects/4d8bcc347dc54dd9910820347aaf7bda => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:13.832396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.833740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:13.847398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899a95-a255-4f66-b35b-40f461951e61 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.852166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.852786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.853066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.853341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] PATH_INFO: `/v3/users/ea8952533e924ae086059dc74c52c673` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.854479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8952533e924ae086059dc74c52c673)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.856960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.899934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1103/2245] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:13 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:13.904791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.906432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:13.918991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec92200-8c0e-499a-8e2b-1fdbcb5a9c21 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.927786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.929097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.929097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.929343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.929952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.931805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:13.951090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1104/2246] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:13.951090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:13.958243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d798d-9dbc-4f5c-9185-24f844b5c213 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:13.965883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:13.966710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:13.966970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:13.967251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] PATH_INFO: `/v3/projects/f52a6510a7654c89a5b5cc5801e772f0/users/242ff24d3c8f48d996ff67bb8272741f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:13.988758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52a6510a7654c89a5b5cc5801e772f0, user_id=242ff24d3c8f48d996ff67bb8272741f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:13.994677 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:14.132476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1105/2247] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:13 2022] PUT /identity/v3/projects/f52a6510a7654c89a5b5cc5801e772f0/users/242ff24d3c8f48d996ff67bb8272741f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:14.132692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:14.133841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:14.141576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8952533e924ae086059dc74c52c673'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:14.142315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8952533e924ae086059dc74c52c673'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:14.142963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8952533e924ae086059dc74c52c673'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:14.154317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8952533e924ae086059dc74c52c673'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:14.170468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1ea083b8-860d-4180-b868-1350e7cabb45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8952533e924ae086059dc74c52c673'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:14.213651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df2664-7f33-451b-beb6-58b1b2d48a26 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:14.219043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:14.219657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:14.219938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:14.220212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:14.220908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:14.222490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:14.233209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1106/2248] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:14.237390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:14.239083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:14.250965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1143/2249] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:13 2022] DELETE /identity/v3/users/ea8952533e924ae086059dc74c52c673 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:14.256253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a63e6-b922-4fb3-a082-e01dcf816c13 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:14.307978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:14.319588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:14.320245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:14.320525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:14.320849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] PATH_INFO: `/v3/projects/f52a6510a7654c89a5b5cc5801e772f0/users/242ff24d3c8f48d996ff67bb8272741f/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:14.322920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52a6510a7654c89a5b5cc5801e772f0, user_id=242ff24d3c8f48d996ff67bb8272741f, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:14.326276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:14.336854 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1107/2250] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:14 2022] PUT /identity/v3/projects/f52a6510a7654c89a5b5cc5801e772f0/users/242ff24d3c8f48d996ff67bb8272741f/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:14.350869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:14.351175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:14.351445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:14.381232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] MFA Rules not processed for user `242ff24d3c8f48d996ff67bb8272741f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:14.382450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:14.462286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1144/2251] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:14.479594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:14.481586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:14.543506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74223fb-67c7-4ab0-a22a-2eb4caae7b2a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:14.547126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:14.547729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:14.548001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:14.549646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:14.557657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:14.559183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:14.569074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1108/2252] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:14.708619 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:14.710219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:14.786853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63267738-88bd-4285-b8ce-36b31b5c812d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:14.791431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:14.792065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:14.792358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:14.793237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] PATH_INFO: `/v3/projects/819929ac35b649f0980a0935e204ae2e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:14.794355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819929ac35b649f0980a0935e204ae2e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:14.796283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:14.813487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:14.836412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:14.845423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490119-8783-439b-8264-2688853fb090 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:14.849104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:14.849695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:14.849971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:14.852882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:14.852882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:14.916639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:14.916639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:14.920809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819929ac35b649f0980a0935e204ae2e', 'request_id': 'req-b9318efa-ea40-46a5-ad3c-91f001934b74'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:14.927452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1109/2253] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:15.019893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:15.095278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819929ac35b649f0980a0935e204ae2e', 'request_id': 'req-b9318efa-ea40-46a5-ad3c-91f001934b74'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.102204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819929ac35b649f0980a0935e204ae2e', 'request_id': 'req-b9318efa-ea40-46a5-ad3c-91f001934b74'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.113208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] The token cache is being invalidate because project 819929ac35b649f0980a0935e204ae2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:15.113577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.114296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.115881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1145/2254] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:14 2022] DELETE /identity/v3/projects/819929ac35b649f0980a0935e204ae2e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:15.124724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:15.136881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d3afb-7622-4020-9226-70a675381c34 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:15.198790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:15.199429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:15.199702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:15.199971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] PATH_INFO: `/v3/users/1dcaf1e5eb804e8e8f9d5147f89e72f2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:15.201134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dcaf1e5eb804e8e8f9d5147f89e72f2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:15.203278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:15.439673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dcaf1e5eb804e8e8f9d5147f89e72f2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.440368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dcaf1e5eb804e8e8f9d5147f89e72f2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.441194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dcaf1e5eb804e8e8f9d5147f89e72f2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.451872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dcaf1e5eb804e8e8f9d5147f89e72f2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.469775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63e6f111-e0ac-4475-a30f-31c96b743e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dcaf1e5eb804e8e8f9d5147f89e72f2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:15.551655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:15.560788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1110/2255] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:15 2022] DELETE /identity/v3/users/1dcaf1e5eb804e8e8f9d5147f89e72f2 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:16.040464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:16.041952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:16.059006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318efa-ea40-46a5-ad3c-91f001934b74 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:16.120496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:16.122045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.122331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.122643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] PATH_INFO: `/v3/projects/88a3ad13e32b4ab9af05d7fd559b8ddb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.123815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a3ad13e32b4ab9af05d7fd559b8ddb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:16.125713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:16.229545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a3ad13e32b4ab9af05d7fd559b8ddb', 'request_id': 'req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:16.323768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a3ad13e32b4ab9af05d7fd559b8ddb', 'request_id': 'req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:16.331541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a3ad13e32b4ab9af05d7fd559b8ddb', 'request_id': 'req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:16.343235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] The token cache is being invalidate because project 88a3ad13e32b4ab9af05d7fd559b8ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:16.343532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:16.344199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-745cf1bd-602a-49cb-acbc-14c4f8d4d47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:16.345713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1146/2256] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:16 2022] DELETE /identity/v3/projects/88a3ad13e32b4ab9af05d7fd559b8ddb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:16.350696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.350976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.351271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.374049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:16.375294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:16.442095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1111/2257] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:16.451385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.451812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.452158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.472863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:16.474089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:16.483339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1147/2258] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:16.487952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:16.498796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed959af-bd15-48b2-b238-b7d30c70c13a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:16.504870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:16.505499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.505783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.506056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.506746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:16.507054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:16.508761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:16.525637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1112/2259] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:16:16.527605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:16.537440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068229d-8b9f-429d-9ff5-7b8390d09400 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:16.543015 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:16.543720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.544067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.544433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.545158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:16.547295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:16.677611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1148/2260] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:16 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:16.682208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:16.693890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea74a4-ddcd-43e7-8695-8e0a6a3637e9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:16.811414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:16.812038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.812318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.812593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.813237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:16.815488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:16.934651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1113/2261] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:16 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:16.937188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:16.951021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177a235-3660-49d3-b3ab-fad27d6d064c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:16.956694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:16.957781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:16.958168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:16.958452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:16.959299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:16.960904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:16.972849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1149/2262] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:16.987551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:17.001247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4203-ca9b-41d9-b1bb-56727fef3121 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:17.008942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64711ed2-5eea-4774-895e-f8417c7d99ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:17.009903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64711ed2-5eea-4774-895e-f8417c7d99ba admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:17.009903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64711ed2-5eea-4774-895e-f8417c7d99ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:17.010083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64711ed2-5eea-4774-895e-f8417c7d99ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:17.010773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64711ed2-5eea-4774-895e-f8417c7d99ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:17.013418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64711ed2-5eea-4774-895e-f8417c7d99ba admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:17.030652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1114/2263] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:17.033618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:17.045690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8171c-d1ec-4fa4-90cf-b97e95dd146b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:17.050720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:17.051450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:17.051732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:17.052010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] PATH_INFO: `/v3/projects/702737b15c8e484d8aba627563179edf/users/54bff8714b5b4f0d96b8fbeda69e4e98/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:17.069601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702737b15c8e484d8aba627563179edf, user_id=54bff8714b5b4f0d96b8fbeda69e4e98, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:17.072306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:17.075180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:17.223020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1150/2264] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:17 2022] PUT /identity/v3/projects/702737b15c8e484d8aba627563179edf/users/54bff8714b5b4f0d96b8fbeda69e4e98/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:17.228514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26e657-7568-487d-be60-c17668663617 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:17.228770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26e657-7568-487d-be60-c17668663617 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:17.229036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26e657-7568-487d-be60-c17668663617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:17.248918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9d26e657-7568-487d-be60-c17668663617 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:17.250106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d26e657-7568-487d-be60-c17668663617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:17.313249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1115/2265] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:17.322953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:17.324785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:17.385267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa36e-971a-4f3b-abf1-c3bd1688d334 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:17.389299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1148408-acde-4c8f-968f-4bddf9f74ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:17.389915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1148408-acde-4c8f-968f-4bddf9f74ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:17.390198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1148408-acde-4c8f-968f-4bddf9f74ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:17.390474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1148408-acde-4c8f-968f-4bddf9f74ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:17.400728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1148408-acde-4c8f-968f-4bddf9f74ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:17.402919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1148408-acde-4c8f-968f-4bddf9f74ac2 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:17.413750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1151/2266] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:17.610241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26e657-7568-487d-be60-c17668663617 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:17.611760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d26e657-7568-487d-be60-c17668663617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:17.626199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:17.630593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26e657-7568-487d-be60-c17668663617 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:17.644594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:17.647505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:17.647505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:17.647505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] PATH_INFO: `/v3/users/aaa7278dd81f44c7989b0abe55fb600c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:17.647505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa7278dd81f44c7989b0abe55fb600c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:17.648907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:17.965909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa7278dd81f44c7989b0abe55fb600c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:17.966624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa7278dd81f44c7989b0abe55fb600c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:17.967231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa7278dd81f44c7989b0abe55fb600c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:17.974733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa7278dd81f44c7989b0abe55fb600c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:17.987595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa7278dd81f44c7989b0abe55fb600c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:18.096139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1116/2267] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:17 2022] DELETE /identity/v3/users/aaa7278dd81f44c7989b0abe55fb600c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:18.100865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:18.101350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:18.101431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:18.121292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:18.129418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:18.193838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1152/2268] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:18.205561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:18.207292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:18.279616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecc322-d133-48cc-aa23-11e222a0be7d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:18.283593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eb00b-4277-48be-9cf1-d50baeb56bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:18.284258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eb00b-4277-48be-9cf1-d50baeb56bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:18.284535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eb00b-4277-48be-9cf1-d50baeb56bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:18.284815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eb00b-4277-48be-9cf1-d50baeb56bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:18.297006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563eb00b-4277-48be-9cf1-d50baeb56bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:18.298688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563eb00b-4277-48be-9cf1-d50baeb56bb4 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:18.308778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1117/2269] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:18.827993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:18.829511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:18.850326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761d11c-f97a-4cc2-922c-44d20ec10dd9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:18.857100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:18.867352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:18.867741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:18.868202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] PATH_INFO: `/v3/projects/0a15ac32f8d9422c95795fb54745a10a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:18.870233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a15ac32f8d9422c95795fb54745a10a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:18.883278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.006998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a15ac32f8d9422c95795fb54745a10a', 'request_id': 'req-6b1deca5-de28-44a9-abf7-4260965aa651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:19.029692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.030188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.030188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.062362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] MFA Rules not processed for user `54bff8714b5b4f0d96b8fbeda69e4e98`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:19.064258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:19.107744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a15ac32f8d9422c95795fb54745a10a', 'request_id': 'req-6b1deca5-de28-44a9-abf7-4260965aa651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:19.115774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a15ac32f8d9422c95795fb54745a10a', 'request_id': 'req-6b1deca5-de28-44a9-abf7-4260965aa651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:19.128234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] The token cache is being invalidate because project 0a15ac32f8d9422c95795fb54745a10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:19.128572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:19.129209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b1deca5-de28-44a9-abf7-4260965aa651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:19.131028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1153/2270] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:18 2022] DELETE /identity/v3/projects/0a15ac32f8d9422c95795fb54745a10a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:19.136270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef393-8b44-4f60-81ed-79e8e858a0ad None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.136558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef393-8b44-4f60-81ed-79e8e858a0ad None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.136833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef393-8b44-4f60-81ed-79e8e858a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.145932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1118/2271] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:19.159649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.161186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:19.161422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b46ef393-8b44-4f60-81ed-79e8e858a0ad None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:19.162668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b46ef393-8b44-4f60-81ed-79e8e858a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:19.173007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:19.176933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:19.225201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1154/2272] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:19.232354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d90a0a-e1fe-4d2e-842c-0326a08b4be6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.232354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.232354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.232354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.232675 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.233843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.234134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.234414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.236066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:19.248179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.249647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.254493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:19.255737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:19.258815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1119/2273] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:19.264746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1155/2274] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:19.268343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.280300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021dc90-c67a-490a-abca-875f4f004119 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.285098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.285701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.285980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.286252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.286970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:19.289169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.289169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.301917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1120/2275] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:16:19.306149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.316291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd7332-ef28-4a29-88f2-68aefba2c75c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.321202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.321819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.322099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.322405 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.323093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.325027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.410508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1156/2276] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:19 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:19.414603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.426306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d788d5b-54e2-4a16-a5ab-6e2be574cb64 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.488794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.489582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.489879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.490179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.490944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.493815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.615295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1121/2277] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:19 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:19.619421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.630269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d653e-92d0-4f0b-b215-fa5b0524014b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.637301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.637961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.638326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.638529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.639187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.640915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.657408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1157/2278] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:19.660450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.671079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825859b-c603-446e-81df-877a2b87d425 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.677684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d98d28-e241-4fed-976d-9ce2cc26d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.677684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98d28-e241-4fed-976d-9ce2cc26d253 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.677684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98d28-e241-4fed-976d-9ce2cc26d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.677684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98d28-e241-4fed-976d-9ce2cc26d253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.677973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d98d28-e241-4fed-976d-9ce2cc26d253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.679666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d98d28-e241-4fed-976d-9ce2cc26d253 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.699828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1122/2279] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:19.703401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:19.715141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d73c66-2c6a-4413-ab01-67ad98f8c46d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:19.722351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:19.722351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.723303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.723303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] PATH_INFO: `/v3/projects/beb7afeb0df846f487c9284cd1ff3f7a/users/85e7e4646e964619a21f2adbf453fde6/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.739369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb7afeb0df846f487c9284cd1ff3f7a, user_id=85e7e4646e964619a21f2adbf453fde6, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:19.742204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:19.851736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1158/2280] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:19 2022] PUT /identity/v3/projects/beb7afeb0df846f487c9284cd1ff3f7a/users/85e7e4646e964619a21f2adbf453fde6/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:19.862613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:19.862923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:19.863204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:19.911594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] MFA Rules not processed for user `85e7e4646e964619a21f2adbf453fde6`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:19.915318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:20.033763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1123/2281] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:20.036123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.127802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5c8e3-05f9-4b84-9240-148f077239ba admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:20.133644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:20.134177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.134456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.134760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.135398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:20.137834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:20.242471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1159/2282] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:20 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:20.247047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.257930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5667c1-a436-4be2-87b3-b43016d896d0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:20.262217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:20.317272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:20.317855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.318121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.318447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.319247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:20.321154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:20.422127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1124/2283] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:20 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:20.425692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.437188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e071d58-6816-498b-972d-4d6bc8f1a9bc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:20.443612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:20.445546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.445546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.445546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.445797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:20.447440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:20.463164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1160/2284] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:20.470426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.482889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4928c8-3f24-4355-8214-ba331ac13ecd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:20.487957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8bc44-e5d3-42e9-9872-2a2cac3644b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:20.488642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8bc44-e5d3-42e9-9872-2a2cac3644b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.489098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8bc44-e5d3-42e9-9872-2a2cac3644b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.489430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8bc44-e5d3-42e9-9872-2a2cac3644b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.490154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8bc44-e5d3-42e9-9872-2a2cac3644b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:20.491915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8bc44-e5d3-42e9-9872-2a2cac3644b2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:20.503207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1125/2285] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:20.507223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.519739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b647-cc88-416f-8113-6621b7362b14 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:20.525463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:20.526119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.526426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.526714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] PATH_INFO: `/v3/projects/68cf65dcfa4f4e60baa724d264b80eb2/users/f9216a86cc774bceaf1d021bd7517174/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.545388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cf65dcfa4f4e60baa724d264b80eb2, user_id=f9216a86cc774bceaf1d021bd7517174, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:20.548135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:20.626208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1161/2286] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:20 2022] PUT /identity/v3/projects/68cf65dcfa4f4e60baa724d264b80eb2/users/f9216a86cc774bceaf1d021bd7517174/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:20.637714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.637996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.638292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.670787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] MFA Rules not processed for user `f9216a86cc774bceaf1d021bd7517174`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:20.672068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:20.760352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1126/2287] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:20.765102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.774450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:20.829211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad51d2-3072-4f99-b092-e956c13e5a5b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:20.834336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:20.835006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:20.835285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:20.835560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:20.836200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:20.838100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:20.939092 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1162/2288] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:20 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:20.941133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:20.954016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5bc71-0d0d-4ea2-a301-b45100ea299d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:21.012370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200937dd-e77d-4022-9a87-3e89116db312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:21.012978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200937dd-e77d-4022-9a87-3e89116db312 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.013258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200937dd-e77d-4022-9a87-3e89116db312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.013528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200937dd-e77d-4022-9a87-3e89116db312 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.014160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200937dd-e77d-4022-9a87-3e89116db312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:21.016140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200937dd-e77d-4022-9a87-3e89116db312 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:21.135011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1127/2289] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:20 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:21.139289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:21.181273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003ae3-7f77-44fc-9237-82db5f1149b5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:21.181992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d106581-a455-4654-ac37-82f1022f081f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.182291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d106581-a455-4654-ac37-82f1022f081f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.182591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d106581-a455-4654-ac37-82f1022f081f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.189568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:21.190417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.190917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.196024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.196024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:21.196209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:21.196290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:21.205885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1163/2290] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:21.207660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5d106581-a455-4654-ac37-82f1022f081f None None] MFA Rules not processed for user `54bff8714b5b4f0d96b8fbeda69e4e98`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:21.215617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5d106581-a455-4654-ac37-82f1022f081f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:21.217626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:21.229809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8aadc-9bad-40d9-bd76-4e7612fec60d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:21.259423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:21.259423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.259423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.259423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] PATH_INFO: `/v3/projects/70f1710c62c64f1a92be10c5c1034f14/users/fb903489e18642778bf311325663f62a/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.299527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f1710c62c64f1a92be10c5c1034f14, user_id=fb903489e18642778bf311325663f62a, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:21.302218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:21.322604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1128/2291] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:21.420147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1164/2292] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:21 2022] PUT /identity/v3/projects/70f1710c62c64f1a92be10c5c1034f14/users/fb903489e18642778bf311325663f62a/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:21.424229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d106581-a455-4654-ac37-82f1022f081f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:21.440333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d106581-a455-4654-ac37-82f1022f081f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:21.500373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801bcc6-ee67-4b2f-b4f5-95e06ad9cdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:21.500991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801bcc6-ee67-4b2f-b4f5-95e06ad9cdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.501269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801bcc6-ee67-4b2f-b4f5-95e06ad9cdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.501537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801bcc6-ee67-4b2f-b4f5-95e06ad9cdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.502179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801bcc6-ee67-4b2f-b4f5-95e06ad9cdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:21.503871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801bcc6-ee67-4b2f-b4f5-95e06ad9cdc9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:21.514088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1129/2293] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:21.518121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:21.529529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e910d9-52ec-4091-9fbf-f909a39ce952 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:21.534580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:21.535334 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.535613 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.535884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] PATH_INFO: `/v3/projects/70f1710c62c64f1a92be10c5c1034f14/users/fb903489e18642778bf311325663f62a/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.537898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f1710c62c64f1a92be10c5c1034f14, user_id=fb903489e18642778bf311325663f62a, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:21.540556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:21.552852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1165/2294] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:21 2022] PUT /identity/v3/projects/70f1710c62c64f1a92be10c5c1034f14/users/fb903489e18642778bf311325663f62a/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:21.563389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.563934 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.564213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.593535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] MFA Rules not processed for user `fb903489e18642778bf311325663f62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:21.594705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:21.666294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1130/2295] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:21.678355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:21.680690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:21.746391 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e121db-01d0-4f4f-9981-640c351032f8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:21.750346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:21.751014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:21.752263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:21.752263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:21.818692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:21.820511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:21.824996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:21.831847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1166/2296] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:26.088977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:26.090762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:26.103560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb082602-77df-4f4e-8ebb-212d811dc24a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:26.107613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:26.108220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:26.112126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:26.112126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:26.141398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:26.186000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:26.188668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:26.206370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1131/2297] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:28.127660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:28.137540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629105e-c482-4762-96e7-6d6643639f2c service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:28.147142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa148d-2f4d-480c-bf71-f7a1461f5db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:28.148490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa148d-2f4d-480c-bf71-f7a1461f5db7 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:28.150872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa148d-2f4d-480c-bf71-f7a1461f5db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:28.151228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa148d-2f4d-480c-bf71-f7a1461f5db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:28.162161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa148d-2f4d-480c-bf71-f7a1461f5db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:28.163732 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa148d-2f4d-480c-bf71-f7a1461f5db7 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:28.179996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1167/2298] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:31.800368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:31.801874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:31.877893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c06a6d-c7cd-405b-b609-d897bea2e82e service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:31.882809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:31.883443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:31.883721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:31.884347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] PATH_INFO: `/v3/users/bd31c08d6e034ea3a3534ffe6561d246` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:31.885655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd31c08d6e034ea3a3534ffe6561d246)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:31.887723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:32.148020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd31c08d6e034ea3a3534ffe6561d246'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:32.148613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd31c08d6e034ea3a3534ffe6561d246'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:32.149214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd31c08d6e034ea3a3534ffe6561d246'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:32.157232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd31c08d6e034ea3a3534ffe6561d246'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:32.170264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd31c08d6e034ea3a3534ffe6561d246'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:32.259758 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1132/2299] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:31 2022] DELETE /identity/v3/users/bd31c08d6e034ea3a3534ffe6561d246 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:32.264784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:32.265219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:32.265328 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:32.285252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:32.286466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:32.344641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1168/2300] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:32.354135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:32.355549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:32.408181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:32.411863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77ac2a-c6d3-4ceb-bbf6-455b6c0b7e33 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:32.415500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:32.418569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:32.418569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:32.418569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:32.425455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:32.426914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:32.436091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1133/2301] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:32.866799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:32.876242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:32.880116 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837a8d1-9b54-4a43-8ea7-b165eb7f2d72 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:32.884709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:32.885352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:32.885627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:32.885898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] PATH_INFO: `/v3/projects/0de3278f10da45dfb6f53ef75e922c84` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:32.887064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de3278f10da45dfb6f53ef75e922c84)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:32.888968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:33.044436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de3278f10da45dfb6f53ef75e922c84', 'request_id': 'req-a88c5f05-f4e6-41a2-9129-c016808ee727'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.151624 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de3278f10da45dfb6f53ef75e922c84', 'request_id': 'req-a88c5f05-f4e6-41a2-9129-c016808ee727'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.159521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de3278f10da45dfb6f53ef75e922c84', 'request_id': 'req-a88c5f05-f4e6-41a2-9129-c016808ee727'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.170674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] The token cache is being invalidate because project 0de3278f10da45dfb6f53ef75e922c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:33.171746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.171746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.173345 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1169/2302] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:32 2022] DELETE /identity/v3/projects/0de3278f10da45dfb6f53ef75e922c84 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:33.177261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:33.187807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42523ff-a76f-42c8-a9bf-e556a0c0583d service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:33.267122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:33.267748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:33.268025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:33.268294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] PATH_INFO: `/v3/users/242ff24d3c8f48d996ff67bb8272741f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:33.269400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] RBAC: Authorizing `identity:delete_user(user_id=242ff24d3c8f48d996ff67bb8272741f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:33.271376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:33.463088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242ff24d3c8f48d996ff67bb8272741f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.463844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242ff24d3c8f48d996ff67bb8272741f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.464576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242ff24d3c8f48d996ff67bb8272741f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.472048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242ff24d3c8f48d996ff67bb8272741f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.484040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-07289700-53c1-4f08-b491-b5ab0ef0481d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242ff24d3c8f48d996ff67bb8272741f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:33.554970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1134/2303] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:33 2022] DELETE /identity/v3/users/242ff24d3c8f48d996ff67bb8272741f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:34.070572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.072795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:34.088493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c5f05-f4e6-41a2-9129-c016808ee727 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.151529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.152326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.153209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.153618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] PATH_INFO: `/v3/projects/f52a6510a7654c89a5b5cc5801e772f0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.154897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52a6510a7654c89a5b5cc5801e772f0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:34.156897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:34.167892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:34.296445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52a6510a7654c89a5b5cc5801e772f0', 'request_id': 'req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:34.368840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52a6510a7654c89a5b5cc5801e772f0', 'request_id': 'req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:34.380519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52a6510a7654c89a5b5cc5801e772f0', 'request_id': 'req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:34.396326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] The token cache is being invalidate because project f52a6510a7654c89a5b5cc5801e772f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:34.401288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:34.401288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3863ce90-93e8-4ba5-99d7-73fbd7015aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:34.408527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1170/2304] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:34 2022] DELETE /identity/v3/projects/f52a6510a7654c89a5b5cc5801e772f0 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:34.413451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.413939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.414673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.435692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:34.436905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:34.503017 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1135/2305] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:34.512481 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.513673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.513969 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.537052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:34.538457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:34.548075 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1171/2306] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:34.553926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.567350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787465a1-3a95-4f12-8825-29f3ac36b541 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.572221 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.572975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.573325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.573828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.575115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:34.575492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:34.577399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:34.584580 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:34.589921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1136/2307] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:16:34.595640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.615992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ead9a-94d7-450b-9b66-98174c8a5c94 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.622404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.623044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.623326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.623602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.624251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:34.626182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:34.712753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1172/2308] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:34 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:34.717505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.727642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5674c-b4d4-4985-9af4-b5e60d95128a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.791307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.791962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.792234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.797431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.797431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:34.797431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:34.883808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1137/2309] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:34 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:34.888272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.911410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595583c-5ba3-4f3c-add3-7e28c28ef2b5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.916585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.917249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.917535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.917846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.918532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:34.920188 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:34.932127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1173/2310] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:34.936891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.948691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20640a-f423-49e7-8218-e038eb6d0457 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.955139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb16261-41e4-4667-b194-74bccf471d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.956139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb16261-41e4-4667-b194-74bccf471d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.956575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb16261-41e4-4667-b194-74bccf471d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.957142 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb16261-41e4-4667-b194-74bccf471d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:34.957857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb16261-41e4-4667-b194-74bccf471d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:34.959776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb16261-41e4-4667-b194-74bccf471d1d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:34.974045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1138/2311] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:34.977972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:34.988746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1680a8-02d2-47a4-9953-4aabb1f8bf7b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:34.998050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:34.998936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:34.999351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:34.999701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] PATH_INFO: `/v3/projects/76f697fc1e134999a18afacbc230b8fd/users/13d5f36a124a4058bf4ee951443917f1/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.020307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f697fc1e134999a18afacbc230b8fd, user_id=13d5f36a124a4058bf4ee951443917f1, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.023017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.089034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1174/2312] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:34 2022] PUT /identity/v3/projects/76f697fc1e134999a18afacbc230b8fd/users/13d5f36a124a4058bf4ee951443917f1/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:35.105144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.105144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.105144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.130999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] MFA Rules not processed for user `13d5f36a124a4058bf4ee951443917f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:35.132253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:35.217398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1139/2313] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:35.222197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.310625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c547-f44e-404a-95a9-d45b275b88a7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:35.322753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:35.322753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.322753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.322753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.323330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.323438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.405205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1175/2314] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:35 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:35.410430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.422222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eac73b-c492-433a-a0b1-b741016732b0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:35.500422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:35.501145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.502716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.502716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.502716 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.504503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.517549 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:35.603510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1140/2315] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:35 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:35.608424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.621114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eb0ae-c7bd-46df-8bb4-754456782e8c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:35.627434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:35.628348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.628788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.629063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.629714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.640867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.647521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1176/2316] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:35.648893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.659848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb4840-1929-4824-bd9a-53420f83fc06 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:35.665011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:35.665681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.669626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.669626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] PATH_INFO: `/v3/projects/001dc685c6d0405cb06e7afcd30b822b/users/5cd45b10da31443eae743004b1a33392/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.684813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001dc685c6d0405cb06e7afcd30b822b, user_id=5cd45b10da31443eae743004b1a33392, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.687768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.781070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1141/2317] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:35 2022] PUT /identity/v3/projects/001dc685c6d0405cb06e7afcd30b822b/users/5cd45b10da31443eae743004b1a33392/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:35.785258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.796105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a206d-8fba-4d81-8bde-f661940dc65b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:35.801315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:35.854365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834ed21-1d18-4aaf-a386-c20a897e650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:35.855019 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834ed21-1d18-4aaf-a386-c20a897e650f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.855294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834ed21-1d18-4aaf-a386-c20a897e650f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.855564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834ed21-1d18-4aaf-a386-c20a897e650f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.856240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834ed21-1d18-4aaf-a386-c20a897e650f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.857830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834ed21-1d18-4aaf-a386-c20a897e650f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.868239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1177/2318] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:35.873109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.884387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496cfb-fcff-4633-a2f3-4ec54aee9b22 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:35.889866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:35.890637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.890976 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.891321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] PATH_INFO: `/v3/projects/001dc685c6d0405cb06e7afcd30b822b/users/5cd45b10da31443eae743004b1a33392/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.895335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] RBAC: Authorizing `identity:create_grant(project_id=001dc685c6d0405cb06e7afcd30b822b, user_id=5cd45b10da31443eae743004b1a33392, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:35.899806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:35.914353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1142/2319] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:35 2022] PUT /identity/v3/projects/001dc685c6d0405cb06e7afcd30b822b/users/5cd45b10da31443eae743004b1a33392/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:35.924862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:35.925147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:35.925456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:35.956437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] MFA Rules not processed for user `5cd45b10da31443eae743004b1a33392`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:35.957828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:35.994222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:35.999466 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:36.039932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1178/2320] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:36.051145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:36.052570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:36.073747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11d01-f65c-4cfc-8766-f5a775a9649b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:36.086351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:36.087449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:36.088276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:36.088276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] PATH_INFO: `/v3/users/85e7e4646e964619a21f2adbf453fde6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:36.094226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e7e4646e964619a21f2adbf453fde6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:36.094226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:36.144908 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd282654-a0dd-4df7-bacb-9e848c2310fe None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:36.150335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791640-4fd4-46ef-bf3e-39ac224da1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:36.151055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791640-4fd4-46ef-bf3e-39ac224da1dc service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:36.151341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791640-4fd4-46ef-bf3e-39ac224da1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:36.151798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791640-4fd4-46ef-bf3e-39ac224da1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:36.162630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791640-4fd4-46ef-bf3e-39ac224da1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:36.164059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791640-4fd4-46ef-bf3e-39ac224da1dc service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:36.177282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1179/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:36.345041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e7e4646e964619a21f2adbf453fde6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:36.345913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e7e4646e964619a21f2adbf453fde6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:36.346672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e7e4646e964619a21f2adbf453fde6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:36.356166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e7e4646e964619a21f2adbf453fde6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:36.372717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e7e4646e964619a21f2adbf453fde6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:36.481601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1143/2322] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:35 2022] DELETE /identity/v3/users/85e7e4646e964619a21f2adbf453fde6 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:36.488252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:36.488779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:36.489259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:36.517930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:36.517930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:36.560692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:36.611931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1180/2323] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:36.617901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:36.621555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:36.689514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:36.709084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c925d-40c1-4be6-8e25-73a504978bd9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:36.713294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:36.714005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:36.714505 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:36.715054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:36.728619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:36.730216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:36.750518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1144/2324] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:37.317616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:37.319207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:37.346469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c39a7-b0ea-477f-944e-b5bdd919335f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:37.352824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:37.356295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:37.356963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:37.357569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] PATH_INFO: `/v3/projects/beb7afeb0df846f487c9284cd1ff3f7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:37.359217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb7afeb0df846f487c9284cd1ff3f7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:37.361650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:37.508004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb7afeb0df846f487c9284cd1ff3f7a', 'request_id': 'req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.571732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:37.573812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:37.590789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd7d8-40e9-4126-af18-00f02cff4ba4 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:37.620453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb7afeb0df846f487c9284cd1ff3f7a', 'request_id': 'req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.632785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb7afeb0df846f487c9284cd1ff3f7a', 'request_id': 'req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.653638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] The token cache is being invalidate because project beb7afeb0df846f487c9284cd1ff3f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:37.654132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.654826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.656616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1181/2325] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:37 2022] DELETE /identity/v3/projects/beb7afeb0df846f487c9284cd1ff3f7a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:37.660534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:37.661890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:37.666005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:37.666680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:37.666961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:37.667238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] PATH_INFO: `/v3/users/3acd0aed6f9946c5b895341708347ec6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:37.669048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3acd0aed6f9946c5b895341708347ec6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:37.671408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:37.675816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7f81e-e2c6-4e7f-a5cc-9c4a7a410702 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:37.680637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:37.681338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:37.681710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:37.681898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] PATH_INFO: `/v3/users/f9216a86cc774bceaf1d021bd7517174` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:37.683228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9216a86cc774bceaf1d021bd7517174)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:37.687566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:37.932151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9216a86cc774bceaf1d021bd7517174'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.932151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9216a86cc774bceaf1d021bd7517174'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.936064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3acd0aed6f9946c5b895341708347ec6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.936911 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3acd0aed6f9946c5b895341708347ec6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.937756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3acd0aed6f9946c5b895341708347ec6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.946294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9216a86cc774bceaf1d021bd7517174'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.951613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3acd0aed6f9946c5b895341708347ec6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.966108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3acd0aed6f9946c5b895341708347ec6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:37.973627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9216a86cc774bceaf1d021bd7517174'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.002649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9216a86cc774bceaf1d021bd7517174'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.129954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1145/2326] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:37 2022] DELETE /identity/v3/users/3acd0aed6f9946c5b895341708347ec6 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:38.197659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1182/2327] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:37 2022] DELETE /identity/v3/users/f9216a86cc774bceaf1d021bd7517174 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:38.504582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:38.506071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:38.595212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5ead-a547-47c6-8415-0ac93bdf5f2d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:38.602766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:38.672291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:38.672921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:38.673251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:38.673629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] PATH_INFO: `/v3/projects/acc8d05179964efb9c80f5d33c74cd3f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:38.675579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc8d05179964efb9c80f5d33c74cd3f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:38.686387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:38.686491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:38.687327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:38.704097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed0510-809c-48da-a099-d242e617db0d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:38.714824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:38.714824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:38.714824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:38.714824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] PATH_INFO: `/v3/projects/68cf65dcfa4f4e60baa724d264b80eb2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:38.715810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cf65dcfa4f4e60baa724d264b80eb2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:38.718935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:38.755849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:38.804414 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc8d05179964efb9c80f5d33c74cd3f', 'request_id': 'req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.811742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc8d05179964efb9c80f5d33c74cd3f', 'request_id': 'req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.817403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc8d05179964efb9c80f5d33c74cd3f', 'request_id': 'req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.837884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cf65dcfa4f4e60baa724d264b80eb2', 'request_id': 'req-53b300b7-1de2-4ab7-a1b2-361b1dccc199'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.915014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] The token cache is being invalidate because project acc8d05179964efb9c80f5d33c74cd3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:38.915304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.915939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.917461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1146/2328] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:38 2022] DELETE /identity/v3/projects/acc8d05179964efb9c80f5d33c74cd3f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:38.921227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:38.922487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:38.968902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cf65dcfa4f4e60baa724d264b80eb2', 'request_id': 'req-53b300b7-1de2-4ab7-a1b2-361b1dccc199'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.978275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cf65dcfa4f4e60baa724d264b80eb2', 'request_id': 'req-53b300b7-1de2-4ab7-a1b2-361b1dccc199'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.989113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94af7-d2dc-4d8f-8462-7bd6e40e9cd8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:38.993178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] The token cache is being invalidate because project 68cf65dcfa4f4e60baa724d264b80eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:38.993480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.994251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:38.996724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1183/2329] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:38 2022] DELETE /identity/v3/projects/68cf65dcfa4f4e60baa724d264b80eb2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:39.000524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:39.002329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:39.046486 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:39.047159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:39.047439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:39.047750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] PATH_INFO: `/v3/users/15f5587fe7224113b285d38e6fd37416` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:39.049449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f5587fe7224113b285d38e6fd37416)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:39.051491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:39.068821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b300b7-1de2-4ab7-a1b2-361b1dccc199 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:39.074081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:39.074497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:39.074810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:39.075086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] PATH_INFO: `/v3/users/fb903489e18642778bf311325663f62a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:39.076230 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb903489e18642778bf311325663f62a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:39.078144 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:39.231753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:39.259727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f5587fe7224113b285d38e6fd37416'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.261335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f5587fe7224113b285d38e6fd37416'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.262042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f5587fe7224113b285d38e6fd37416'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.262171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb903489e18642778bf311325663f62a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.267212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb903489e18642778bf311325663f62a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.268244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb903489e18642778bf311325663f62a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.277743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f5587fe7224113b285d38e6fd37416'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.281945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb903489e18642778bf311325663f62a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.305961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb903489e18642778bf311325663f62a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.308969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f5587fe7224113b285d38e6fd37416'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:39.391980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1184/2330] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:38 2022] DELETE /identity/v3/users/fb903489e18642778bf311325663f62a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:39.461600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1147/2331] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:38 2022] DELETE /identity/v3/users/15f5587fe7224113b285d38e6fd37416 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:39.789509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:39.791065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:39.851667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:39.882264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43175eaf-5110-43a8-a102-e46a2935a7cc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:39.905422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:39.908379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:39.965210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:39.965210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:39.965210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:39.965210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] PATH_INFO: `/v3/projects/f527746e49b24de4aa96b69822a477ff` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:39.966529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f527746e49b24de4aa96b69822a477ff)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:39.968998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.016094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c9c73-d822-43e3-a333-d9ea47d08fda admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.020935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.021558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.021835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.022109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] PATH_INFO: `/v3/projects/70f1710c62c64f1a92be10c5c1034f14` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.023255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f1710c62c64f1a92be10c5c1034f14)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.025517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.114868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f527746e49b24de4aa96b69822a477ff', 'request_id': 'req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.183915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f1710c62c64f1a92be10c5c1034f14', 'request_id': 'req-4cb684c2-5d98-48b5-ba25-03775eb8fa04'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.190140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f1710c62c64f1a92be10c5c1034f14', 'request_id': 'req-4cb684c2-5d98-48b5-ba25-03775eb8fa04'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.195651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f527746e49b24de4aa96b69822a477ff', 'request_id': 'req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.197729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f1710c62c64f1a92be10c5c1034f14', 'request_id': 'req-4cb684c2-5d98-48b5-ba25-03775eb8fa04'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.200892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:40.206838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f527746e49b24de4aa96b69822a477ff', 'request_id': 'req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.214165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] The token cache is being invalidate because project f527746e49b24de4aa96b69822a477ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:40.214459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.215142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.216703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1185/2332] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:39 2022] DELETE /identity/v3/projects/f527746e49b24de4aa96b69822a477ff => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:40.217563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.218866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:40.284493 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7b6ae-78ce-463d-a2c0-eb098012c6c1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.340460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.341208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.341537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.341814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] PATH_INFO: `/v3/users/13d5f36a124a4058bf4ee951443917f1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.342987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d5f36a124a4058bf4ee951443917f1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.344919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.359352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] The token cache is being invalidate because project 70f1710c62c64f1a92be10c5c1034f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:40.359743 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.360606 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4cb684c2-5d98-48b5-ba25-03775eb8fa04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.374588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1148/2333] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:39 2022] DELETE /identity/v3/projects/70f1710c62c64f1a92be10c5c1034f14 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:40.374588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4414-4d4e-4f47-9e53-8147d8531b50 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.374588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4414-4d4e-4f47-9e53-8147d8531b50 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.374588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4414-4d4e-4f47-9e53-8147d8531b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.387268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-264d4414-4d4e-4f47-9e53-8147d8531b50 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:40.389086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-264d4414-4d4e-4f47-9e53-8147d8531b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:40.398270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1149/2334] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:40.407802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.408083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.408354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.430037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:40.431307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:40.440273 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1150/2335] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:40.444639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.454723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ce69-80ce-4c5a-91eb-0c3c70a77c30 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.460227 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.460858 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.461139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.461413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.462163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:40.462527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.464201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.478212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1151/2336] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:16:40.483025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.493756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d03c5e-9bd0-4216-8bef-e8489535dd05 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.501328 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.501958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.502239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.504833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.504833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.505373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.581362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d5f36a124a4058bf4ee951443917f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.582028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d5f36a124a4058bf4ee951443917f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.582677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d5f36a124a4058bf4ee951443917f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.592164 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:40.594661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d5f36a124a4058bf4ee951443917f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.613171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06ff9616-8245-4070-b2e8-e33e78967a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d5f36a124a4058bf4ee951443917f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:40.639450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1152/2337] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:40 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:40.644973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.647813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:40.689167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1186/2338] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:40 2022] DELETE /identity/v3/users/13d5f36a124a4058bf4ee951443917f1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:40.694027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.694418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.694799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.718190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:40.720597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:40.726420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c29d3-29de-426b-8f9a-f6fcb4a61cc6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.803130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.803706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1187/2339] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:40.804210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.807031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.807389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.807967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.810754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.813851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.815225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:40.873051 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2434d-9290-4ba3-b70c-54126101b0e1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.876790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.877394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.877675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.877949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.887270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.888845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1153/2340] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:40 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:40.889295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.892789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.899149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1188/2341] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:40.903002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2601e-71cb-4f15-aedb-b66dc28b050c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.907736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.908383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.908658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.908937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.909579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.911302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.921852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1154/2342] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:40.925720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.935762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b769-f3ec-40af-a90e-44267cedf51f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.940865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f822ed-a248-41dd-9736-abfbea836742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.941496 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f822ed-a248-41dd-9736-abfbea836742 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.941782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f822ed-a248-41dd-9736-abfbea836742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.942057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f822ed-a248-41dd-9736-abfbea836742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.942814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f822ed-a248-41dd-9736-abfbea836742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.944419 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f822ed-a248-41dd-9736-abfbea836742 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:40.955289 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1189/2343] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:40.959309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:40.969670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95c1c-3797-4c22-a6b6-d74eaf73a374 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:40.974655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:40.975313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:40.975595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:40.975870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] PATH_INFO: `/v3/projects/c3122dd3663643e49d828798942f46a5/users/e99112744829446eaff3da49c70eb51c/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:40.985837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:40.991650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3122dd3663643e49d828798942f46a5, user_id=e99112744829446eaff3da49c70eb51c, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:40.994310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.140709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1155/2344] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:40 2022] PUT /identity/v3/projects/c3122dd3663643e49d828798942f46a5/users/e99112744829446eaff3da49c70eb51c/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:41.152244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.152554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.152868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.185521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] MFA Rules not processed for user `e99112744829446eaff3da49c70eb51c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:41.186774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:41.274692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1190/2345] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:41.279797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.289222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:41.343780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734523-6a31-4c66-a9b1-43dfde166e3e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.348657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.349258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.349537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.349809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.350436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.352390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.421642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.423286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:41.447125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8c88c-e05d-46c1-8055-3ede1dd8e37a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.452572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.453206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.453487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.453868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] PATH_INFO: `/v3/projects/76f697fc1e134999a18afacbc230b8fd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.454913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f697fc1e134999a18afacbc230b8fd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.457255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.472569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1156/2346] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:41 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:41.473722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.535037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057f99-ef9f-4217-a3a3-0cdb559e425a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.539587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.540182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.540455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.540725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.541351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.543319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.569343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f697fc1e134999a18afacbc230b8fd', 'request_id': 'req-a976a449-1f8a-4b6e-b021-3551bd163264'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.640864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1157/2347] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:41 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:41.644585 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.655228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bb0a-69dd-419d-a380-16756ce5ac9c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.660341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f697fc1e134999a18afacbc230b8fd', 'request_id': 'req-a976a449-1f8a-4b6e-b021-3551bd163264'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.668557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f697fc1e134999a18afacbc230b8fd', 'request_id': 'req-a976a449-1f8a-4b6e-b021-3551bd163264'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.685070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] The token cache is being invalidate because project 76f697fc1e134999a18afacbc230b8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:41.685783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.686668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.703466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1191/2348] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:41 2022] DELETE /identity/v3/projects/76f697fc1e134999a18afacbc230b8fd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:41.703466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.703466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:41.718352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.718992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.719268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.719540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.720186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.721860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.735060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1158/2349] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:41.736054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.737733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:41.744507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:41.749692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112ba95-d719-4cd0-88ca-ed1d322742d8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.754452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.756150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.756150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.756150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.756410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.757895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.765115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976a449-1f8a-4b6e-b021-3551bd163264 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.770053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.770226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1159/2350] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:41.771108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.771552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.773355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] PATH_INFO: `/v3/users/5cd45b10da31443eae743004b1a33392` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.773355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd45b10da31443eae743004b1a33392)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.774531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.775303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.784557 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ac21a-9726-4a0b-a313-e928a56783da admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:41.789310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f98721-7645-4368-9cbf-9b3e66aa9435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:41.789882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98721-7645-4368-9cbf-9b3e66aa9435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.790155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98721-7645-4368-9cbf-9b3e66aa9435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.790454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98721-7645-4368-9cbf-9b3e66aa9435 admin admin] PATH_INFO: `/v3/projects/f4055aaaffa74eb58fe180a7b386fe40/users/b001f6b1efb340459eb6911f08236709/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.807703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f98721-7645-4368-9cbf-9b3e66aa9435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4055aaaffa74eb58fe180a7b386fe40, user_id=b001f6b1efb340459eb6911f08236709, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:41.809275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:41.812786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f98721-7645-4368-9cbf-9b3e66aa9435 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:41.868192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1160/2351] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:41 2022] PUT /identity/v3/projects/f4055aaaffa74eb58fe180a7b386fe40/users/b001f6b1efb340459eb6911f08236709/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:41.878962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:41.879243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:41.879513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:41.912329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] MFA Rules not processed for user `b001f6b1efb340459eb6911f08236709`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:41.913561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:41.937260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd45b10da31443eae743004b1a33392'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.938288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd45b10da31443eae743004b1a33392'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.940680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd45b10da31443eae743004b1a33392'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.950800 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd45b10da31443eae743004b1a33392'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.967902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd45b10da31443eae743004b1a33392'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:41.983304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1161/2352] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:41.987643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:41.989259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:42.076433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408f1b-484e-486c-8cc3-058cda63696d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.081733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.082431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1192/2353] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:41 2022] DELETE /identity/v3/users/5cd45b10da31443eae743004b1a33392 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:42.082599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.083020 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.083362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.084059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.086024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.217874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1162/2354] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:41 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:42.219868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.318461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eadb6-3fb5-4a26-b6e4-776ed0741217 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.381552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.382200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.382497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.382806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.383465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.385402 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.473901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1193/2355] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:42 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:42.475395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.489880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d835f0-acd7-4373-9582-7393ca019f94 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.497153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.497774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.498050 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.498324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.499207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.500826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.512243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1163/2356] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:42.519905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.530981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7f9f-b9c6-4801-9418-59ad9a784582 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.536099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.536742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.537042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.537314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] PATH_INFO: `/v3/projects/a6e4dbc732b74ad5a463507e47fe5b51/users/539b72c8097340168ef8dd9cfcf9ea60/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.553849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e4dbc732b74ad5a463507e47fe5b51, user_id=539b72c8097340168ef8dd9cfcf9ea60, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.556487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.568018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:42.575281 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.577107 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:42.590648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9359-8982-4d2a-832a-54dacefe2a11 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.595664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.596295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.596571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.596844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] PATH_INFO: `/v3/projects/001dc685c6d0405cb06e7afcd30b822b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.597970 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001dc685c6d0405cb06e7afcd30b822b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.599992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.654794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1194/2357] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:42 2022] PUT /identity/v3/projects/a6e4dbc732b74ad5a463507e47fe5b51/users/539b72c8097340168ef8dd9cfcf9ea60/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:42.658704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.722763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3113815-63f0-4e12-b94c-d6b0f98aba3f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.725636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:42.735653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001dc685c6d0405cb06e7afcd30b822b', 'request_id': 'req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:42.746777 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001dc685c6d0405cb06e7afcd30b822b', 'request_id': 'req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:42.753156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001dc685c6d0405cb06e7afcd30b822b', 'request_id': 'req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:42.795649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.796267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.796545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.796818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.797470 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.799140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.809339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1195/2358] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:42.813317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.822833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02836a4-80c2-4e56-a9ff-cc10ece01428 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.823461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] The token cache is being invalidate because project 001dc685c6d0405cb06e7afcd30b822b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:42.823750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:42.824373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b2f86ecf-4ec9-4237-bd74-b3fa2d141bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:42.826387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1164/2359] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:42 2022] DELETE /identity/v3/projects/001dc685c6d0405cb06e7afcd30b822b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:42.828642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac9cc-1f73-4308-b019-9d5a65b4f92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.829284 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac9cc-1f73-4308-b019-9d5a65b4f92a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.829673 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac9cc-1f73-4308-b019-9d5a65b4f92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.829957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac9cc-1f73-4308-b019-9d5a65b4f92a admin admin] PATH_INFO: `/v3/projects/a6e4dbc732b74ad5a463507e47fe5b51/users/539b72c8097340168ef8dd9cfcf9ea60/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.831044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce273d0-f411-446d-88c4-bf8d8b1d0a7e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.831323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce273d0-f411-446d-88c4-bf8d8b1d0a7e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.831592 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce273d0-f411-446d-88c4-bf8d8b1d0a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.831950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac9cc-1f73-4308-b019-9d5a65b4f92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e4dbc732b74ad5a463507e47fe5b51, user_id=539b72c8097340168ef8dd9cfcf9ea60, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.834869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac9cc-1f73-4308-b019-9d5a65b4f92a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.846085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1196/2360] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:42 2022] PUT /identity/v3/projects/a6e4dbc732b74ad5a463507e47fe5b51/users/539b72c8097340168ef8dd9cfcf9ea60/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:42.851880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3ce273d0-f411-446d-88c4-bf8d8b1d0a7e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:42.853085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3ce273d0-f411-446d-88c4-bf8d8b1d0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:42.858890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.859303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.859583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.889935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] MFA Rules not processed for user `539b72c8097340168ef8dd9cfcf9ea60`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:42.891008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:42.916093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1165/2361] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:42.923763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e39f2-358c-4638-9d38-22f54289300e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.924053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e39f2-358c-4638-9d38-22f54289300e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.924331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e39f2-358c-4638-9d38-22f54289300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.944387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-736e39f2-358c-4638-9d38-22f54289300e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:42.945593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-736e39f2-358c-4638-9d38-22f54289300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:42.954078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1166/2362] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:42.958244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e39f2-358c-4638-9d38-22f54289300e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.964821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1197/2363] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:42.969194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e39f2-358c-4638-9d38-22f54289300e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:42.973965 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:42.974620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:42.974908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:42.975196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:42.975843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:16:42.976153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:42.977742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:42.977858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:42.979292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:42.993697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1167/2364] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:16:42.994684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.005811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58690e2-c645-4f10-a710-125cc9404f9e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.011047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.011798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.012085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.012436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.013078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.015081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.043803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c74d5d-c0d2-4014-9d8e-81d2d86a6d51 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.048162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.048821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.056099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.056099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.056099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:43.118229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1168/2365] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:42 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:43.122702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.127234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.128750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.142922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1198/2366] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:43.197809 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:43.205314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3a181-048d-4bed-b0a9-eab2aba7ed97 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.210582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.216475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.216475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.216475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.216475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.216475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.317480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1169/2367] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:43 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:16:43.321422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.337089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b30c6-34c2-425b-a20f-c8a8df46266b service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.342356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.342957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.343236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.343813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.353004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.353004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.355648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1199/2368] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:43.360400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.370383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca5aa-3019-4d2f-897b-a7f6630db4a4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.375099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10fc9-701d-4601-aed8-712923123f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.375687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10fc9-701d-4601-aed8-712923123f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.375968 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10fc9-701d-4601-aed8-712923123f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.376258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10fc9-701d-4601-aed8-712923123f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.376879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10fc9-701d-4601-aed8-712923123f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.378462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10fc9-701d-4601-aed8-712923123f0e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.388329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1170/2369] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:16:43.392385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.402442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e690fd0-44e3-4584-afda-35d3c1f14596 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.407716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.408361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.408637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.408909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] PATH_INFO: `/v3/projects/b0f0480e723a4653afbec0a051609b1f/users/e21abb157f40436d904216bc80771b84/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.425695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f0480e723a4653afbec0a051609b1f, user_id=e21abb157f40436d904216bc80771b84, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.428383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.511571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1200/2370] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:43 2022] PUT /identity/v3/projects/b0f0480e723a4653afbec0a051609b1f/users/e21abb157f40436d904216bc80771b84/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:43.524006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.524538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.524808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.541980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:43.561174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] MFA Rules not processed for user `e21abb157f40436d904216bc80771b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:43.562742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:43.638928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1171/2371] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:43.643469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.729995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97131c5d-5b11-4364-8d6b-3de12e48b0e7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.734753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.740746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.740746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.740746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.740746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.740746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.749484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:43.875124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.878104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1201/2372] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:16:43 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:43.878236 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:43.879248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:43.893625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4cced-0d05-4ae3-a0ff-58322ccc8f41 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.967356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e947af2-38ee-4754-8cae-264c03f44257 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:43.973273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.973885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.974170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.974434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.975123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.977280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:43.984381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:43.985190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:43.985781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:43.985995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] PATH_INFO: `/v3/users/4eec5bc744b54cac82f94ea84ec90eeb` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:43.987632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eec5bc744b54cac82f94ea84ec90eeb)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:43.989745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.112856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1202/2373] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:16:43 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:16:44.117163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:44.127810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa4740-e66e-41fb-a155-4797b1aaf0aa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:44.132671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:44.133249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.133527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.133797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.134429 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:44.136085 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.146990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1203/2374] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:44.151250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:44.160620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638451ef-3ec9-47cc-a544-1ceb6d42457c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:44.165741 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:44.166384 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.166928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.167205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] PATH_INFO: `/v3/projects/cb69d9d4d0b24954b55fd558063d4cea/users/0f1b7e4fed5b4873bb169cc200efbd88/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.182811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb69d9d4d0b24954b55fd558063d4cea, user_id=0f1b7e4fed5b4873bb169cc200efbd88, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:44.186076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.211256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eec5bc744b54cac82f94ea84ec90eeb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:44.212190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eec5bc744b54cac82f94ea84ec90eeb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:44.212822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eec5bc744b54cac82f94ea84ec90eeb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:44.219993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eec5bc744b54cac82f94ea84ec90eeb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:44.232362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55748dc9-a833-4d26-9675-d26801423e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eec5bc744b54cac82f94ea84ec90eeb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:44.284451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1204/2375] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:44 2022] PUT /identity/v3/projects/cb69d9d4d0b24954b55fd558063d4cea/users/0f1b7e4fed5b4873bb169cc200efbd88/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:44.288231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:44.289831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:44.310674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1172/2376] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:16:43 2022] DELETE /identity/v3/users/4eec5bc744b54cac82f94ea84ec90eeb => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:44.315809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.316162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.316507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.335738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:44.336974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:44.369531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34a5c8-613f-4d72-b6a6-af4abefa4468 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:44.421561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1173/2377] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:44.438946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:44.438946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:44.439424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:44.451300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:44.452189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.452585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.453178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.453836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:44.456027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.461516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:44.469375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1205/2378] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:16:44.472968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:44.489056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec2947-329b-4a6c-99d3-0c85c8d50050 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:44.520086 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796c818-f033-4d7b-b82f-a98c4133b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:44.520958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796c818-f033-4d7b-b82f-a98c4133b816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.521365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796c818-f033-4d7b-b82f-a98c4133b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.525652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796c818-f033-4d7b-b82f-a98c4133b816 admin admin] PATH_INFO: `/v3/projects/cb69d9d4d0b24954b55fd558063d4cea/users/0f1b7e4fed5b4873bb169cc200efbd88/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.531476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796c818-f033-4d7b-b82f-a98c4133b816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb69d9d4d0b24954b55fd558063d4cea, user_id=0f1b7e4fed5b4873bb169cc200efbd88, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:44.537887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342dfa8-73c3-4eff-b8c7-23b65cb1d177 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:44.538401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796c818-f033-4d7b-b82f-a98c4133b816 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.542437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:44.543105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.547936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.547936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.560037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:44.563170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.564057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1206/2379] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:16:44 2022] PUT /identity/v3/projects/cb69d9d4d0b24954b55fd558063d4cea/users/0f1b7e4fed5b4873bb169cc200efbd88/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:16:44.576737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1174/2380] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:44.585446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.585815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.586780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.642806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] MFA Rules not processed for user `0f1b7e4fed5b4873bb169cc200efbd88`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:44.645650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:44.729453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1207/2381] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:16:44.740842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:44.742220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:44.814690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48433e-7325-43d4-b293-07e125fc7274 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:44.875788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:44.876339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:44.876660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:44.876940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:44.880768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:44.953658 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:44.955178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:44.965095 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1175/2382] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:45.040593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:45.042105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:45.165283 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742bcc0-9ae9-47b0-b6f4-5f2dad044ee9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:45.170282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:45.171838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:45.172491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:45.172740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] PATH_INFO: `/v3/projects/5482356a0cf74dc995decac42546f883` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:45.173895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5482356a0cf74dc995decac42546f883)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:45.176671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:45.320524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5482356a0cf74dc995decac42546f883', 'request_id': 'req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:45.400871 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:45.404305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:45.413443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e3a5a-6910-4717-bdbe-bb9f42c2cb4a service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:45.429034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5482356a0cf74dc995decac42546f883', 'request_id': 'req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:45.440097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5482356a0cf74dc995decac42546f883', 'request_id': 'req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:45.442829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:45.452765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:45.455286 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] The token cache is being invalidate because project 5482356a0cf74dc995decac42546f883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:16:45.455578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:45.456240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:16:45.457801 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1208/2383] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:16:45 2022] DELETE /identity/v3/projects/5482356a0cf74dc995decac42546f883 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:16:45.481121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:45.482705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:45.483054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:45.483392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:45.485239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:45.559023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:45.560587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:45.583190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1176/2384] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:16:47.243865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:47.245252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:47.257905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0470-c8a7-4e46-a90f-b8f7a3a1b382 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:47.261715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:47.263725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:47.263725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:47.263725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:47.265151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:47.342322 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:47.343796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f9fa5-352a-4c15-b5cf-9dddaefea3a7 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:47.354262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1209/2385] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:53.690979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:53.702358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:53.703501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:53.703776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:53.710122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9218c-81a4-4249-9cd8-466a778079a0 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:53.724249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:53.724975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:53.725269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:53.725525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:53.727961 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:53.742126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:53.743880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:53.814055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1210/2386] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:16:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:16:53.815538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:53.817045 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dbbf0-2089-4d63-b4fd-09142eaeb5a7 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:53.827352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1177/2387] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:16:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:16:53.859479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:16:53.860938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:53.934599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20a3ce-a7fb-49e6-838c-3fe4d77201c9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:16:53.949274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:16:53.949274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:53.949274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:53.949274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:53.957057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:16:53.958698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:16:53.972331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1211/2388] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:16:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 19:16:56.022008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea773b-d760-4d78-893f-90e8777f2481 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:16:56.022532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea773b-d760-4d78-893f-90e8777f2481 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:16:56.022532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea773b-d760-4d78-893f-90e8777f2481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:16:56.068294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5aea773b-d760-4d78-893f-90e8777f2481 None None] MFA Rules not processed for user `54bff8714b5b4f0d96b8fbeda69e4e98`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:16:56.069865 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5aea773b-d760-4d78-893f-90e8777f2481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:16:56.090784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:16:56.156657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1178/2389] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:00.189249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:00.190813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:00.220326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:00.292124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0353-d51d-454b-a8f4-eb42f4643ff4 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:00.295042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:00.295684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:00.295965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:00.296238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] PATH_INFO: `/v3/users/e99112744829446eaff3da49c70eb51c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:00.297372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99112744829446eaff3da49c70eb51c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:00.299318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:00.497566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99112744829446eaff3da49c70eb51c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:00.498300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99112744829446eaff3da49c70eb51c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:00.498929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99112744829446eaff3da49c70eb51c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:00.506685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99112744829446eaff3da49c70eb51c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:00.523878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99112744829446eaff3da49c70eb51c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:00.609714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1212/2390] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:00 2022] DELETE /identity/v3/users/e99112744829446eaff3da49c70eb51c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:00.614284 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:00.615067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:00.615283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:00.635487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:00.636775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:00.718471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1179/2391] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:00.734011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:00.735730 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:00.825472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ed8fa-ca0a-42fa-9c3a-441f19f88182 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:00.833861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:00.834483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:00.834837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:00.835052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:00.850332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:00.851805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:00.867715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1213/2392] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:01.774780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:01.778324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:01.803618 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a929377-dc6c-4b08-a640-4baee1f2ca06 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:01.825725 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:01.829977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:01.830359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:01.830525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] PATH_INFO: `/v3/projects/c3122dd3663643e49d828798942f46a5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:01.831955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3122dd3663643e49d828798942f46a5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:01.833878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:01.966921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3122dd3663643e49d828798942f46a5', 'request_id': 'req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:01.977006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3122dd3663643e49d828798942f46a5', 'request_id': 'req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:01.984780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3122dd3663643e49d828798942f46a5', 'request_id': 'req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.095239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] The token cache is being invalidate because project c3122dd3663643e49d828798942f46a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:02.095739 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.102373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.104624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1180/2393] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:01 2022] DELETE /identity/v3/projects/c3122dd3663643e49d828798942f46a5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:02.108516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:02.126130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246d085-44ed-441f-80c3-82ce6597210e service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:02.230154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:02.258532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:02.259528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:02.259903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:02.260275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] PATH_INFO: `/v3/users/b001f6b1efb340459eb6911f08236709` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:02.261529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b001f6b1efb340459eb6911f08236709)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:02.264112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:02.542670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b001f6b1efb340459eb6911f08236709'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.544281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b001f6b1efb340459eb6911f08236709'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.544995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b001f6b1efb340459eb6911f08236709'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.557218 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b001f6b1efb340459eb6911f08236709'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.596903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b001f6b1efb340459eb6911f08236709'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:02.683511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1214/2394] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:02 2022] DELETE /identity/v3/users/b001f6b1efb340459eb6911f08236709 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:03.004354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:03.006363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:03.033429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6658cf-dc80-4035-a89a-b9e01bfd2f60 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:03.129907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:03.149005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:03.149921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:03.150679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:03.151295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] PATH_INFO: `/v3/users/3ae692f22824423bbfa52429ccf5aa9c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:03.153270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae692f22824423bbfa52429ccf5aa9c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:03.157778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:03.347758 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:03.349268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:03.373433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654dcc-eeb6-4ffc-b089-97884efa8148 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:03.394540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:03.395229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:03.395506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:03.398212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] PATH_INFO: `/v3/projects/f4055aaaffa74eb58fe180a7b386fe40` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:03.402997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4055aaaffa74eb58fe180a7b386fe40)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:03.404955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:03.437225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae692f22824423bbfa52429ccf5aa9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.438449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae692f22824423bbfa52429ccf5aa9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.439621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae692f22824423bbfa52429ccf5aa9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.449202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae692f22824423bbfa52429ccf5aa9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.477665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-87e6109b-7362-4289-91b5-1299fd6e70be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae692f22824423bbfa52429ccf5aa9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.551237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4055aaaffa74eb58fe180a7b386fe40', 'request_id': 'req-b3239f1e-0231-4d49-bbc3-2852bd849686'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.595891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1181/2395] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:03 2022] DELETE /identity/v3/users/3ae692f22824423bbfa52429ccf5aa9c => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:03.600085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:03.600616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:03.600966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:03.623498 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:03.624333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:03.653720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4055aaaffa74eb58fe180a7b386fe40', 'request_id': 'req-b3239f1e-0231-4d49-bbc3-2852bd849686'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.672497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4055aaaffa74eb58fe180a7b386fe40', 'request_id': 'req-b3239f1e-0231-4d49-bbc3-2852bd849686'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.707586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] The token cache is being invalidate because project f4055aaaffa74eb58fe180a7b386fe40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:03.709041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.709797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.713623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1182/2396] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:03.720335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1215/2397] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:03 2022] DELETE /identity/v3/projects/f4055aaaffa74eb58fe180a7b386fe40 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:03.724264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:03.724394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:03.725876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:03.726733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:03.739197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad74b6-0570-4f6c-8493-d8c66824e0a8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:03.744642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:03.745484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:03.745599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:03.745805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] PATH_INFO: `/v3/users/539b72c8097340168ef8dd9cfcf9ea60` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:03.747010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=539b72c8097340168ef8dd9cfcf9ea60)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:03.749041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:03.823563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:03.853357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3239f1e-0231-4d49-bbc3-2852bd849686 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:03.942988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:03.943850 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:03.943950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:03.944196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:03.948394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:03.952154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '539b72c8097340168ef8dd9cfcf9ea60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.953186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '539b72c8097340168ef8dd9cfcf9ea60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.954002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '539b72c8097340168ef8dd9cfcf9ea60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.966034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '539b72c8097340168ef8dd9cfcf9ea60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.972185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:03.979698 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:03.981436 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:03.983604 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '539b72c8097340168ef8dd9cfcf9ea60'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:03.995575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1216/2398] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:04.070429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1183/2399] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:03 2022] DELETE /identity/v3/users/539b72c8097340168ef8dd9cfcf9ea60 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:04.504174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:04.506819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:04.555904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d219eb-ac3a-4da8-8324-552f5789d91a service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:04.657005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:04.657005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:04.659920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:04.660573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:04.660851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:04.661151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] PATH_INFO: `/v3/projects/aade1725a943407b9baf6f3eb1fdb46f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:04.664369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aade1725a943407b9baf6f3eb1fdb46f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:04.665432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:04.683106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e1b43-19f5-4d6b-9646-79c7f369fcbe admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:04.693522 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:04.694938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:04.695408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:04.695755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] PATH_INFO: `/v3/projects/a6e4dbc732b74ad5a463507e47fe5b51` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:04.697875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e4dbc732b74ad5a463507e47fe5b51)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:04.700455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:04.849579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aade1725a943407b9baf6f3eb1fdb46f', 'request_id': 'req-ebb41703-156d-49bb-9bf8-e568532031b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:04.910239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e4dbc732b74ad5a463507e47fe5b51', 'request_id': 'req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:04.916935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e4dbc732b74ad5a463507e47fe5b51', 'request_id': 'req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:04.939276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e4dbc732b74ad5a463507e47fe5b51', 'request_id': 'req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:04.995446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aade1725a943407b9baf6f3eb1fdb46f', 'request_id': 'req-ebb41703-156d-49bb-9bf8-e568532031b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:05.003199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aade1725a943407b9baf6f3eb1fdb46f', 'request_id': 'req-ebb41703-156d-49bb-9bf8-e568532031b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:05.014602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] The token cache is being invalidate because project aade1725a943407b9baf6f3eb1fdb46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:05.014898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:05.015556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ebb41703-156d-49bb-9bf8-e568532031b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:05.017064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1217/2400] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:04 2022] DELETE /identity/v3/projects/aade1725a943407b9baf6f3eb1fdb46f => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:05.022231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e233ec2-7997-4529-86d4-20ccecb3da73 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.027375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e233ec2-7997-4529-86d4-20ccecb3da73 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.027701 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e233ec2-7997-4529-86d4-20ccecb3da73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.051715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6e233ec2-7997-4529-86d4-20ccecb3da73 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:05.051972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e233ec2-7997-4529-86d4-20ccecb3da73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:05.111269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] The token cache is being invalidate because project a6e4dbc732b74ad5a463507e47fe5b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:05.111269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:05.111269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-637d2d6d-fe24-4e3b-b0b0-dc210ad5cd62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:05.111269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1184/2401] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:04 2022] DELETE /identity/v3/projects/a6e4dbc732b74ad5a463507e47fe5b51 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:17:05.113358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:05.123424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df5df1-6722-4ca8-a55d-c0d497ec7756 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.158764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df5df1-6722-4ca8-a55d-c0d497ec7756 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.158764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df5df1-6722-4ca8-a55d-c0d497ec7756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.158764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-16df5df1-6722-4ca8-a55d-c0d497ec7756 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:05.158764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-16df5df1-6722-4ca8-a55d-c0d497ec7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:05.179239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1185/2402] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:05.181078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.181387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.181649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.185785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1218/2403] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:05.195162 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.195386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.196252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.205047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:05.210630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:05.217141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1186/2404] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:05.226719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:05.230262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:05.233986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:05.237704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cf1dc-ddbd-4758-a6a9-e2abb23bb1e5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:05.244269 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:05.245085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.245616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.245882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1219/2405] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:05.246776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.247514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:17:05.247825 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:05.249450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:05.253269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:05.263544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d5480-3943-42be-ab4d-8de75cb2206e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:05.265691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1187/2406] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:17:05.268895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:05.269687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.269823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:05.269978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.270282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.271070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:17:05.271375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:05.272980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:05.282653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a81fa-0a90-42ea-a93d-7fb9895c7696 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:05.290358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1220/2407] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:17:05.290821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:05.290821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.290821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.290821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.291215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:05.291505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:05.292121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:05.301505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d843de1-db20-4a38-8fd3-46df3aa2429c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:05.320677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:05.321300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.321578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.321859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.323625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:05.325172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:05.729529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1188/2408] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/projects => generated 365 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:17:05.734100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:05.782243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1221/2409] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/projects => generated 375 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:05.786950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:05.819170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489729-b87a-4d43-9f96-9518691dfe5b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:05.854619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:05.888335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce44db7-469e-4d8c-93b7-1261e3892828 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:05.893739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:05.895637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.895637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.895637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.909460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:05.914731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:05.945842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:05.946511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:05.946868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:05.947099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:05.947703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:05.950291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.119339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1189/2410] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/users => generated 283 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:17:06.124194 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.137929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803059e3-3464-4f46-834f-ff80d1cafcb7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.146066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.146585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1222/2411] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:05 2022] POST /identity/v3/users => generated 288 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:06.146714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.146985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.147307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.147970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.148113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.150755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.169403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1190/2412] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:06.170640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.180795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdffc7-db43-4026-9699-ab75e9cf7cda admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.181426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3b42f-ecd5-4a94-9013-2003a36e2836 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.187781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.188405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.188732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.189007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.189656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.191451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.201426 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.203149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.206940 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1191/2413] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:06.207360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.215072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.215528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.215528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.215528 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.219681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064597ff-7b02-4010-b44d-7ca86905d349 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.225013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e43e0b-ee95-4ba2-9e94-a4d05373b391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.225850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e43e0b-ee95-4ba2-9e94-a4d05373b391 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.226257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e43e0b-ee95-4ba2-9e94-a4d05373b391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.226646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e43e0b-ee95-4ba2-9e94-a4d05373b391 admin admin] PATH_INFO: `/v3/projects/e6712212ae18438abf5d4c5cf8f16d00/users/09483da8f7094da2b51f6d246cec7e46/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.245168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1223/2414] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:06.248534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.257763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e43e0b-ee95-4ba2-9e94-a4d05373b391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6712212ae18438abf5d4c5cf8f16d00, user_id=09483da8f7094da2b51f6d246cec7e46, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.261844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e43e0b-ee95-4ba2-9e94-a4d05373b391 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.268121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f35a-d2b1-4fe2-92a1-fc9054c485d7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.273791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.274684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.275084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.275455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.276202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.278485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.292981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1224/2415] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:06.294898 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.305687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99442b0-2a07-46ce-81f6-da1c9d245866 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.311022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3fd43-8819-4f94-80a0-3c7c3c9d1649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.311861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3fd43-8819-4f94-80a0-3c7c3c9d1649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.312430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3fd43-8819-4f94-80a0-3c7c3c9d1649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.312853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3fd43-8819-4f94-80a0-3c7c3c9d1649 admin admin] PATH_INFO: `/v3/projects/9ed3c601ba0b44f69cd8f72a427c0a15/users/90163f93ce1e42fe80dd0ff40c079731/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.333541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3fd43-8819-4f94-80a0-3c7c3c9d1649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed3c601ba0b44f69cd8f72a427c0a15, user_id=90163f93ce1e42fe80dd0ff40c079731, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.336851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3fd43-8819-4f94-80a0-3c7c3c9d1649 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.351517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1192/2416] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:06 2022] PUT /identity/v3/projects/e6712212ae18438abf5d4c5cf8f16d00/users/09483da8f7094da2b51f6d246cec7e46/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:06.362338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.362937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.363392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.428817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] MFA Rules not processed for user `09483da8f7094da2b51f6d246cec7e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:06.430064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:06.448190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1225/2417] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:06 2022] PUT /identity/v3/projects/9ed3c601ba0b44f69cd8f72a427c0a15/users/90163f93ce1e42fe80dd0ff40c079731/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:06.458979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.459423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.459620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.574274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] MFA Rules not processed for user `90163f93ce1e42fe80dd0ff40c079731`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:06.575625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:06.589349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1193/2418] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:06.602268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.604220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:06.725124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3f89-554b-43b7-984f-66481ec74bde None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.727770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:06.732584 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1226/2419] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:06.747863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:06.749296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:06.834577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5aa86-d203-44b6-89aa-70d15e7cf398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.838261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5aa86-d203-44b6-89aa-70d15e7cf398 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.838635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5aa86-d203-44b6-89aa-70d15e7cf398 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.838913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5aa86-d203-44b6-89aa-70d15e7cf398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.888395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e0ce7-4b27-49d2-ae12-32ba4b980caf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:06.898093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf4db3-6481-40cf-addc-0569ad566314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:06.901039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf4db3-6481-40cf-addc-0569ad566314 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:06.901329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf4db3-6481-40cf-addc-0569ad566314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:06.901616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf4db3-6481-40cf-addc-0569ad566314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:06.931524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf4db3-6481-40cf-addc-0569ad566314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.933106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf4db3-6481-40cf-addc-0569ad566314 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.957598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1227/2420] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:06.976440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5aa86-d203-44b6-89aa-70d15e7cf398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:06.978539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5aa86-d203-44b6-89aa-70d15e7cf398 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:06.992109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1194/2421] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:07.292780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:09.220954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4cc5b-3e02-498d-840e-283463db0d8e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:09.221457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4cc5b-3e02-498d-840e-283463db0d8e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:09.221827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4cc5b-3e02-498d-840e-283463db0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:09.247236 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5dd4cc5b-3e02-498d-840e-283463db0d8e None None] MFA Rules not processed for user `847f6bef4e7b44f4b0adf1bda9dadb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:09.248465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5dd4cc5b-3e02-498d-840e-283463db0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:09.311919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1228/2422] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:17.553510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172103af-d800-4e89-98eb-ca11dce68a61 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:17.554071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172103af-d800-4e89-98eb-ca11dce68a61 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:17.554071 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172103af-d800-4e89-98eb-ca11dce68a61 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:17.555437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1195/2423] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:17:17.562041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede356-ab32-42c3-bb33-4ac172878cc1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:17.562583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede356-ab32-42c3-bb33-4ac172878cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:17.562583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede356-ab32-42c3-bb33-4ac172878cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:17.704636 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e0ede356-ab32-42c3-bb33-4ac172878cc1 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:17.705858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e0ede356-ab32-42c3-bb33-4ac172878cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:17.787163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1229/2424] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:17:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:17.803753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172103af-d800-4e89-98eb-ca11dce68a61 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:17.806138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-172103af-d800-4e89-98eb-ca11dce68a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:17.898944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172103af-d800-4e89-98eb-ca11dce68a61 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:17.905034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29ea6e-d7ef-4e59-8a66-b5d1a8e2eeb6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:17.906112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29ea6e-d7ef-4e59-8a66-b5d1a8e2eeb6 service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:17.906611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29ea6e-d7ef-4e59-8a66-b5d1a8e2eeb6 service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:17.907223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29ea6e-d7ef-4e59-8a66-b5d1a8e2eeb6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:17.946437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29ea6e-d7ef-4e59-8a66-b5d1a8e2eeb6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:17.950812 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29ea6e-d7ef-4e59-8a66-b5d1a8e2eeb6 service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:17.969267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1196/2425] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:18.248184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd773aa-9555-4244-879d-c7f451cd59dc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:18.248472 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd773aa-9555-4244-879d-c7f451cd59dc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:18.248756 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd773aa-9555-4244-879d-c7f451cd59dc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:18.250329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1230/2426] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:17:18.255895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120f050-17aa-48b4-be94-e863b1dc05d5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:18.256197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120f050-17aa-48b4-be94-e863b1dc05d5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:18.257168 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120f050-17aa-48b4-be94-e863b1dc05d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:18.284254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8120f050-17aa-48b4-be94-e863b1dc05d5 None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:18.286354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8120f050-17aa-48b4-be94-e863b1dc05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:18.348651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1197/2427] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:17:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:18.368523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd773aa-9555-4244-879d-c7f451cd59dc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:18.370830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afd773aa-9555-4244-879d-c7f451cd59dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:18.382293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:18.405468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd773aa-9555-4244-879d-c7f451cd59dc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:18.411571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b454b3-758d-43f2-b90d-5a3c8496f2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:18.414695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b454b3-758d-43f2-b90d-5a3c8496f2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:18.414695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b454b3-758d-43f2-b90d-5a3c8496f2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:18.414695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b454b3-758d-43f2-b90d-5a3c8496f2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:18.426149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b454b3-758d-43f2-b90d-5a3c8496f2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:18.428288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b454b3-758d-43f2-b90d-5a3c8496f2f4 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:18.451208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1231/2428] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:18.553835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05508e-4f99-40ae-afce-144404967aa6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:18.554358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05508e-4f99-40ae-afce-144404967aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:18.554682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05508e-4f99-40ae-afce-144404967aa6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:18.556835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1198/2429] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:17:18.567982 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:18.568550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:18.568714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:18.598688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:18.598688 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:18.611312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1232/2430] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:17:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:18.629185 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05508e-4f99-40ae-afce-144404967aa6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:18.641033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05508e-4f99-40ae-afce-144404967aa6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:18.644718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65043fda-c126-4ab7-b498-540f1019cf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:18.645314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043fda-c126-4ab7-b498-540f1019cf03 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:18.645594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043fda-c126-4ab7-b498-540f1019cf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:18.645868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043fda-c126-4ab7-b498-540f1019cf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:18.657639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65043fda-c126-4ab7-b498-540f1019cf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:18.659201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65043fda-c126-4ab7-b498-540f1019cf03 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:18.671569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1199/2431] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:22.609410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:22.690605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e56cd-32ad-452d-ae7b-57d77a575779 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:22.698954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:22.698954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:22.698954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:22.698954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] PATH_INFO: `/v3/users/90163f93ce1e42fe80dd0ff40c079731` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:22.699319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90163f93ce1e42fe80dd0ff40c079731)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:22.699824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:22.903417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90163f93ce1e42fe80dd0ff40c079731'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:22.904379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90163f93ce1e42fe80dd0ff40c079731'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:22.905287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90163f93ce1e42fe80dd0ff40c079731'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:22.913311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90163f93ce1e42fe80dd0ff40c079731'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:22.928329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90163f93ce1e42fe80dd0ff40c079731'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:23.006887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1233/2432] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:22 2022] DELETE /identity/v3/users/90163f93ce1e42fe80dd0ff40c079731 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:23.012318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:23.012832 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:23.012928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:23.020218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:23.034207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:23.035444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:23.109981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1200/2433] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:23.120056 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:23.121489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:23.182209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bb61-00ec-43b8-b20d-76c95ec03bd0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:23.186090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07262a25-38b3-4e04-a85b-322a4392f61b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:23.186739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07262a25-38b3-4e04-a85b-322a4392f61b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:23.187018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07262a25-38b3-4e04-a85b-322a4392f61b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:23.187288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07262a25-38b3-4e04-a85b-322a4392f61b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:23.197001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07262a25-38b3-4e04-a85b-322a4392f61b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:23.199152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07262a25-38b3-4e04-a85b-322a4392f61b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:23.209665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1234/2434] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:23.658437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:23.659985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:23.677114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f641-183c-49a3-98fc-6b2ff38e7c24 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:23.681808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:23.683087 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:23.683346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:23.683615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] PATH_INFO: `/v3/projects/9ed3c601ba0b44f69cd8f72a427c0a15` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:23.689318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed3c601ba0b44f69cd8f72a427c0a15)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:23.689318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:23.817183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed3c601ba0b44f69cd8f72a427c0a15', 'request_id': 'req-b041ab4b-f4e1-4c2b-8415-4dc92932437f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:23.823290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed3c601ba0b44f69cd8f72a427c0a15', 'request_id': 'req-b041ab4b-f4e1-4c2b-8415-4dc92932437f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:23.828877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed3c601ba0b44f69cd8f72a427c0a15', 'request_id': 'req-b041ab4b-f4e1-4c2b-8415-4dc92932437f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:23.902544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] The token cache is being invalidate because project 9ed3c601ba0b44f69cd8f72a427c0a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:23.902861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:23.903534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b041ab4b-f4e1-4c2b-8415-4dc92932437f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:23.905036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1201/2435] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:23 2022] DELETE /identity/v3/projects/9ed3c601ba0b44f69cd8f72a427c0a15 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:23.915551 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:23.916008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:23.916114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:23.930510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:23.937114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:23.938315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:23.997145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1235/2436] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:24.004916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.005199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.005475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.025365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:24.027090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:24.034957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1202/2437] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:24.039055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.049368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe391-0199-4573-9e2e-5b63c48ee14b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.053885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.054469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.054772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.055038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.055671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:17:24.055963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.057537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.068768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1236/2438] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:17:24.072907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.082146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb707ba-158a-4f61-be47-475aa171a335 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.086919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.087507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.087789 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.088060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.089184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.091123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.190652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1203/2439] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:24 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:24.194799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.204644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afa83d-1d8d-4060-9094-7b2d9f6f7365 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.294259 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.295156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.295433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.295848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.296531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.298443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.386734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1237/2440] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:24 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:17:24.388701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.404336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c0125-aa03-4240-9ad7-34a9d2971ab6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.408921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.409513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.409785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.410055 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.410730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.412321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.422671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1204/2441] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:24.426606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.436462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a37681-ac2f-4ead-9499-f00d5119f966 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.441121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878c7a3-e88a-4e28-97c9-741a23d39806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.441655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878c7a3-e88a-4e28-97c9-741a23d39806 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.441933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878c7a3-e88a-4e28-97c9-741a23d39806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.442205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878c7a3-e88a-4e28-97c9-741a23d39806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.442877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878c7a3-e88a-4e28-97c9-741a23d39806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.444444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878c7a3-e88a-4e28-97c9-741a23d39806 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.455344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1238/2442] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:24.459297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.469775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72b24-fae8-4f13-9286-e53186decaa3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.474349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.475039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.475309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.475583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] PATH_INFO: `/v3/projects/7847d148d828478b8124c8d08c8c582f/users/f9db5f73350c460f99e4ab5cdbe6907a/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.489784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7847d148d828478b8124c8d08c8c582f, user_id=f9db5f73350c460f99e4ab5cdbe6907a, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.492474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.565879 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1205/2443] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:24 2022] PUT /identity/v3/projects/7847d148d828478b8124c8d08c8c582f/users/f9db5f73350c460f99e4ab5cdbe6907a/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:24.571026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.571290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.571586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.592312 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:24.593581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:24.650834 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1239/2444] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:24.661048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:24.662841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:24.689057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:24.721289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef8ff2-4e80-4a59-a17f-bed1ac398a57 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:24.725285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e90d33-fa60-45ee-bda9-f9533335245c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:24.726086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e90d33-fa60-45ee-bda9-f9533335245c service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:24.726369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e90d33-fa60-45ee-bda9-f9533335245c service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:24.726674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e90d33-fa60-45ee-bda9-f9533335245c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:24.736317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e90d33-fa60-45ee-bda9-f9533335245c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:24.737753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e90d33-fa60-45ee-bda9-f9533335245c service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:24.747891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1206/2445] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:26.586432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:26.588131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:26.606127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4741090-830c-473e-b30a-49a3bceb84df None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:26.621200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:26.621854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:26.622127 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:26.622393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] PATH_INFO: `/v3/users/e21abb157f40436d904216bc80771b84` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:26.623628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21abb157f40436d904216bc80771b84)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:26.625535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:26.634140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:26.920764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21abb157f40436d904216bc80771b84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:26.921893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21abb157f40436d904216bc80771b84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:26.922793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21abb157f40436d904216bc80771b84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:26.934642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21abb157f40436d904216bc80771b84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:26.949240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21abb157f40436d904216bc80771b84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:27.038631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1240/2446] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:26 2022] DELETE /identity/v3/users/e21abb157f40436d904216bc80771b84 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:27.043243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:27.044344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:27.045111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:27.067984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:27.069553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:27.167561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1207/2447] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:27.177491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:27.182215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:27.270398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf081-ad02-4df5-8641-ed16a46b66bc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:27.277626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:27.279110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:27.279544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:27.280037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:27.293395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:27.295179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:27.306129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1241/2448] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:27.971842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:27.973514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:27.997103 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f667ea-5d55-466c-abeb-9a0397f4e8f2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:28.009040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:28.009823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:28.010109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:28.012096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] PATH_INFO: `/v3/projects/b0f0480e723a4653afbec0a051609b1f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:28.013271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f0480e723a4653afbec0a051609b1f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:28.016206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:28.169863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f0480e723a4653afbec0a051609b1f', 'request_id': 'req-a78c4ef9-3541-4e98-9146-7b3919191d0e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.177006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f0480e723a4653afbec0a051609b1f', 'request_id': 'req-a78c4ef9-3541-4e98-9146-7b3919191d0e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.192180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f0480e723a4653afbec0a051609b1f', 'request_id': 'req-a78c4ef9-3541-4e98-9146-7b3919191d0e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.287537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] The token cache is being invalidate because project b0f0480e723a4653afbec0a051609b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:28.287537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.287537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.287537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1208/2449] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:27 2022] DELETE /identity/v3/projects/b0f0480e723a4653afbec0a051609b1f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:28.291696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:28.332750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f280e7-4f0c-4284-b40b-c5ccb264f276 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:28.462942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:28.464554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:28.464936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:28.465326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] PATH_INFO: `/v3/users/0f1b7e4fed5b4873bb169cc200efbd88` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:28.466763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1b7e4fed5b4873bb169cc200efbd88)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:28.469076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:28.688193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:28.748030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1b7e4fed5b4873bb169cc200efbd88'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.751986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1b7e4fed5b4873bb169cc200efbd88'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.754689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1b7e4fed5b4873bb169cc200efbd88'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.781094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1b7e4fed5b4873bb169cc200efbd88'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.843281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-375e2865-cb5d-4cb6-b606-d7255fbf6a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1b7e4fed5b4873bb169cc200efbd88'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:28.941928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1242/2450] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:28 2022] DELETE /identity/v3/users/0f1b7e4fed5b4873bb169cc200efbd88 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:29.844257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:29.847462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:29.867331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c4ef9-3541-4e98-9146-7b3919191d0e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:29.921920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:29.935587 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:29.939773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:29.939773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:29.939773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] PATH_INFO: `/v3/projects/cb69d9d4d0b24954b55fd558063d4cea` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:29.939773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb69d9d4d0b24954b55fd558063d4cea)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:29.940399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:30.187162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb69d9d4d0b24954b55fd558063d4cea', 'request_id': 'req-55602252-74bb-4303-87dd-021a5ec7749a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:30.198258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb69d9d4d0b24954b55fd558063d4cea', 'request_id': 'req-55602252-74bb-4303-87dd-021a5ec7749a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:30.208043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb69d9d4d0b24954b55fd558063d4cea', 'request_id': 'req-55602252-74bb-4303-87dd-021a5ec7749a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:30.320130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] The token cache is being invalidate because project cb69d9d4d0b24954b55fd558063d4cea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:30.320336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:30.321279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-55602252-74bb-4303-87dd-021a5ec7749a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:30.323982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1209/2451] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:29 2022] DELETE /identity/v3/projects/cb69d9d4d0b24954b55fd558063d4cea => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:30.328860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8547a1-907f-4889-8100-3577252e986a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:30.329357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8547a1-907f-4889-8100-3577252e986a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:30.329766 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8547a1-907f-4889-8100-3577252e986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:30.354753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cb8547a1-907f-4889-8100-3577252e986a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:30.356546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb8547a1-907f-4889-8100-3577252e986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:30.435559 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1243/2452] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:30.443347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:30.443634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:30.443910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:30.466763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:30.468004 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:30.481190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1210/2453] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:30.481868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8547a1-907f-4889-8100-3577252e986a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:30.497769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8547a1-907f-4889-8100-3577252e986a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:30.516393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:30.517217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:30.517498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:30.517772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:30.518451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:17:30.519253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:30.523975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:30.556459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1244/2454] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:17:30.561323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:30.573008 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed774a09-ef41-4096-9737-9ebea61f88ff None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:30.589101 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:30.589732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:30.590015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:30.590292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:30.590966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:30.592893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:30.724063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1211/2455] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:30 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:30.727999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:30.738666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d8968-e8e0-4f55-b53b-38c4e586cb31 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:30.820692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:30.821318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:30.821662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:30.821939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:30.823295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:30.825352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:30.927689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1245/2456] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:30 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:17:30.935838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:30.952889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e1c20-b421-48f6-8e99-423cb1057ec9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:30.966268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:30.967223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:30.967975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:30.968434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:30.969322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:30.971740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:30.995237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1212/2457] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:17:30.999784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:31.013381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4720c-2dff-45eb-a47d-4237e1ff4191 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:31.029747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5c4bf-58e0-4c27-a216-b67d4abc8bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:31.031552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5c4bf-58e0-4c27-a216-b67d4abc8bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.031552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5c4bf-58e0-4c27-a216-b67d4abc8bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.031552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5c4bf-58e0-4c27-a216-b67d4abc8bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.031552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5c4bf-58e0-4c27-a216-b67d4abc8bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:31.033279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5c4bf-58e0-4c27-a216-b67d4abc8bc2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:31.068010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1246/2458] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:31.075763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:31.078968 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.079454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.083385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.089686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671bc07-2fb1-4e7a-9787-5f8a249cc9a9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:31.099780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2606d-eb03-457f-8b7b-91aebcc78577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:31.101807 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2606d-eb03-457f-8b7b-91aebcc78577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.102260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2606d-eb03-457f-8b7b-91aebcc78577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.103263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2606d-eb03-457f-8b7b-91aebcc78577 admin admin] PATH_INFO: `/v3/projects/de2a8803053642d7b99cdbb4e686f670/users/963c87dd7aa74bcb897633b1affe4fe5/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.115589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:31.116811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:31.123916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2606d-eb03-457f-8b7b-91aebcc78577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2a8803053642d7b99cdbb4e686f670, user_id=963c87dd7aa74bcb897633b1affe4fe5, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:31.127259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2606d-eb03-457f-8b7b-91aebcc78577 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:31.128269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:31.214428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1247/2459] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:17:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:31.214428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:31.214428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:31.284574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1213/2460] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:31 2022] PUT /identity/v3/projects/de2a8803053642d7b99cdbb4e686f670/users/963c87dd7aa74bcb897633b1affe4fe5/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:31.301203 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.302881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.303170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.311327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0985a-edee-48c4-a3f0-4aac60a105af None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:31.387742 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] MFA Rules not processed for user `963c87dd7aa74bcb897633b1affe4fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:31.400193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:31.469808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:31.469808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.469808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.469808 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.522283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1214/2461] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:31.527920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:31.533581 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:31.550210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:31.552118 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:31.566311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1248/2462] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 19:17:31.618701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf9121-4d0e-41f3-a846-6a51bd5b509c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:31.624647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:31.625338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.625621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.625959 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.626627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:31.628621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:31.768379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1215/2463] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:31 2022] POST /identity/v3/projects => generated 367 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:31.772308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:31.785359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe9773-255d-4a1e-bf03-c3ffccdd5aa5 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:31.869759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:31.870987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:31.871447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:31.871957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:31.876878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:31.881110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:32.012211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1249/2464] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:31 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:32.015503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:32.035805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3e06-2f38-44b1-9815-4c0478dcae60 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:32.051046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:32.051784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:32.052239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:32.052762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:32.053644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:32.056014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:32.107524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1216/2465] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:32.112143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:32.169370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004dc4f-7ba1-4099-be6f-b51126b833c6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:32.198916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:32.203987 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:32.204433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:32.208055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] PATH_INFO: `/v3/projects/c2e3829cacb248c2a8cbbf01136b2585/users/4c8ef1d3ebf94e87a3233c2311ac42ec/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:32.278297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3829cacb248c2a8cbbf01136b2585, user_id=4c8ef1d3ebf94e87a3233c2311ac42ec, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:32.295813 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:32.413375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:32.420668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1250/2466] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:32 2022] PUT /identity/v3/projects/c2e3829cacb248c2a8cbbf01136b2585/users/4c8ef1d3ebf94e87a3233c2311ac42ec/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:32.425108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:32.448031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce9ec4-3b60-448c-b4a3-069f60865d03 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:32.638735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04edb-94e4-40b7-942c-93a8223d26fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:32.639373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04edb-94e4-40b7-942c-93a8223d26fa admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:32.639654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04edb-94e4-40b7-942c-93a8223d26fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:32.639929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04edb-94e4-40b7-942c-93a8223d26fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:32.645159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b04edb-94e4-40b7-942c-93a8223d26fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:32.647488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b04edb-94e4-40b7-942c-93a8223d26fa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:32.681343 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1217/2467] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:17:32.693882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:32.724445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be19d51-2260-4033-b20a-37e4b7c171aa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:32.743136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:32.745911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:32.746473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:32.747120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] PATH_INFO: `/v3/projects/c2e3829cacb248c2a8cbbf01136b2585/users/4c8ef1d3ebf94e87a3233c2311ac42ec/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:32.751184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3829cacb248c2a8cbbf01136b2585, user_id=4c8ef1d3ebf94e87a3233c2311ac42ec, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:32.758967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:32.810854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1251/2468] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:32 2022] PUT /identity/v3/projects/c2e3829cacb248c2a8cbbf01136b2585/users/4c8ef1d3ebf94e87a3233c2311ac42ec/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:32.824474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1ef3d-e84e-4e41-8a24-46b7b11f7f9c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:32.825030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1ef3d-e84e-4e41-8a24-46b7b11f7f9c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:32.825617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1ef3d-e84e-4e41-8a24-46b7b11f7f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:32.959782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-37c1ef3d-e84e-4e41-8a24-46b7b11f7f9c None None] MFA Rules not processed for user `4c8ef1d3ebf94e87a3233c2311ac42ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:32.964157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-37c1ef3d-e84e-4e41-8a24-46b7b11f7f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:33.136034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:33.141626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1218/2469] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:33.162812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:33.167275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:33.374477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39927-34b3-465c-9daa-e4c21eebf1fc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:33.383957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:33.384775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:33.385150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:33.385577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:33.409445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:33.412463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:33.440948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1252/2470] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:33.480106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:33.480427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:33.480673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:33.523077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] MFA Rules not processed for user `f9db5f73350c460f99e4ab5cdbe6907a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:33.528387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:33.626065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1219/2471] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:33.632651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:33.644937 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:33.648655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6243fd8-3190-40c1-ba39-46afd9d68d11 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:33.652679 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:33.660367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:33.661024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:33.661300 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:33.662479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:33.810244 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:33.815060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:33.821473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:33.827672 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83774f99-cfda-4deb-9de7-d854f2034b5b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:33.836072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:33.836864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:33.837231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:33.837844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:33.838523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:33.840682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:33.865438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1253/2472] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:33.975342 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1220/2473] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:33 2022] POST /identity/v3/projects => generated 379 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:33.979201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:34.000182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4549-04f2-40ab-9b52-d78d9dfd89db service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:34.190136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:34.190709 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:34.191033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:34.191325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:34.192100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:34.194388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:34.383120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1254/2474] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:33 2022] POST /identity/v3/users => generated 296 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:34.387260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:34.398380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd45b5f-596c-41e6-ab46-45616359c976 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:34.421243 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:34.421875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:34.422152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:34.422426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:34.423128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:34.424791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:34.449399 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1221/2475] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:34.453593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:34.469605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99cf19-d4a5-4f07-8686-a95648e01f25 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:34.479984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:34.480922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:34.482928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:34.482928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] PATH_INFO: `/v3/projects/f07be67668fc4a95a968791b721daec2/users/4511c7c8d2e5433aa1c496c9193ecb32/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:34.505626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07be67668fc4a95a968791b721daec2, user_id=4511c7c8d2e5433aa1c496c9193ecb32, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:34.509148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:34.602531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1255/2476] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:34 2022] PUT /identity/v3/projects/f07be67668fc4a95a968791b721daec2/users/4511c7c8d2e5433aa1c496c9193ecb32/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:17:34.607314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:34.623141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5c534-b0db-4050-9f55-2c51d583d3ff admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:34.739377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:34.740304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:34.740425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:34.740747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:34.741578 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:34.743154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:34.754685 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:34.772220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1222/2477] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:34.776645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:34.790525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5388e3-89a5-4e39-9a2c-975b7c042161 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:34.807395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:34.808079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:34.808643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:34.809126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] PATH_INFO: `/v3/projects/f07be67668fc4a95a968791b721daec2/users/4511c7c8d2e5433aa1c496c9193ecb32/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:34.818814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07be67668fc4a95a968791b721daec2, user_id=4511c7c8d2e5433aa1c496c9193ecb32, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:34.824705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:34.849237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1256/2478] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:34 2022] PUT /identity/v3/projects/f07be67668fc4a95a968791b721daec2/users/4511c7c8d2e5433aa1c496c9193ecb32/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:38.668479 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:38.672451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:39.021363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28abe-fdd8-4dd3-b5fe-a8cc031b0676 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:39.255696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:39.255696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:39.255696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:39.255696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:39.256518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:39.287422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:39.289124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d33ad4-3318-4370-b15a-d752ce26c05e service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:39.314286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1223/2479] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:41.075825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:41.317310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f37928-d63c-4db2-a1f9-2327cb170b4c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:41.326392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:41.327616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:41.328090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:41.328579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] PATH_INFO: `/v3/users/963c87dd7aa74bcb897633b1affe4fe5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:41.330176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963c87dd7aa74bcb897633b1affe4fe5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:41.332639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:41.603420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963c87dd7aa74bcb897633b1affe4fe5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:41.604586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963c87dd7aa74bcb897633b1affe4fe5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:41.607501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963c87dd7aa74bcb897633b1affe4fe5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:41.611290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:41.625387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963c87dd7aa74bcb897633b1affe4fe5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:41.661235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963c87dd7aa74bcb897633b1affe4fe5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:41.757702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1257/2480] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:41 2022] DELETE /identity/v3/users/963c87dd7aa74bcb897633b1affe4fe5 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:41.763210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:41.763794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:41.763980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:41.802453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:41.803906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:41.910048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1224/2481] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:41.920534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:41.922748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:42.069604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda2b1-9e7b-4f35-88a9-64dd67fc4605 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:42.081598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:42.085101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:42.090311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:42.090311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:42.101550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:42.104160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:42.117481 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1258/2482] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:43.064884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:43.066512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:43.092891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788f33-a4dd-4698-890f-924824381e1c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:43.101270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:43.107752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:43.107752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:43.107752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] PATH_INFO: `/v3/projects/de2a8803053642d7b99cdbb4e686f670` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:43.107752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2a8803053642d7b99cdbb4e686f670)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:43.107752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:43.262586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:43.307775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2a8803053642d7b99cdbb4e686f670', 'request_id': 'req-5c7d769d-29c0-44a1-be66-32213539c865'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.324351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2a8803053642d7b99cdbb4e686f670', 'request_id': 'req-5c7d769d-29c0-44a1-be66-32213539c865'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.332533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2a8803053642d7b99cdbb4e686f670', 'request_id': 'req-5c7d769d-29c0-44a1-be66-32213539c865'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.412454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] The token cache is being invalidate because project de2a8803053642d7b99cdbb4e686f670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:43.413648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.415484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.420723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1225/2483] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:43 2022] DELETE /identity/v3/projects/de2a8803053642d7b99cdbb4e686f670 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:43.422082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:43.436805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610698e-dae2-4e1f-8cbe-ad4ac8077c3b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:43.600984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:43.604431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:43.605455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:43.605796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] PATH_INFO: `/v3/users/4c8ef1d3ebf94e87a3233c2311ac42ec` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:43.610110 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c8ef1d3ebf94e87a3233c2311ac42ec)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:43.616206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:43.807687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c8ef1d3ebf94e87a3233c2311ac42ec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.812644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c8ef1d3ebf94e87a3233c2311ac42ec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.812644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c8ef1d3ebf94e87a3233c2311ac42ec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.835740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c8ef1d3ebf94e87a3233c2311ac42ec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.850141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:43.851913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d2dd8af-ab1d-494b-a799-ac622cfa9ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c8ef1d3ebf94e87a3233c2311ac42ec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:43.964479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1259/2484] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:43 2022] DELETE /identity/v3/users/4c8ef1d3ebf94e87a3233c2311ac42ec => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:44.778724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:44.780224 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:44.807524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d769d-29c0-44a1-be66-32213539c865 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:44.940637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:44.941360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:44.941646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:44.941920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] PATH_INFO: `/v3/projects/c2e3829cacb248c2a8cbbf01136b2585` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:44.944169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e3829cacb248c2a8cbbf01136b2585)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:44.947378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.110802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e3829cacb248c2a8cbbf01136b2585', 'request_id': 'req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:45.124897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e3829cacb248c2a8cbbf01136b2585', 'request_id': 'req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:45.131643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e3829cacb248c2a8cbbf01136b2585', 'request_id': 'req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:45.219646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] The token cache is being invalidate because project c2e3829cacb248c2a8cbbf01136b2585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:17:45.221469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:45.222375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ad0bf4ca-10c3-48c7-b64b-6d524d1ba73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:45.224834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1226/2485] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:17:44 2022] DELETE /identity/v3/projects/c2e3829cacb248c2a8cbbf01136b2585 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:17:45.229954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2f54a-3659-4314-a88e-62641e67324c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.230444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2f54a-3659-4314-a88e-62641e67324c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.230536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2f54a-3659-4314-a88e-62641e67324c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.265623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dea2f54a-3659-4314-a88e-62641e67324c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:45.266891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dea2f54a-3659-4314-a88e-62641e67324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:45.319520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.320079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.320533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.356364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] MFA Rules not processed for user `4511c7c8d2e5433aa1c496c9193ecb32`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:45.357339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1260/2486] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:45.357827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:45.365826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.366339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.368784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.391705 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:45.392914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:45.399300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:45.401771 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1261/2487] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:45.406796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.417340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ae8ea-0868-43ce-bd3b-51a5077a4388 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.422630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.423331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.423661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.424917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.424917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:17:45.425355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.426996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.439058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1227/2488] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:45.442583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1262/2489] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:17:45.447936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.455831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.458598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:45.460639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05961a-f205-4ed3-9350-278bfa1de802 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.466179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.466930 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.467516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.467802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.469710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.472244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.538249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16130-77b1-4d14-b9d4-143666fd01fa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.544066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.544671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.544950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.545220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.547198 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:45.586773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1228/2490] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:45 2022] POST /identity/v3/projects => generated 407 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:45.600844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.614439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.616650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.627545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1263/2491] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:45.697225 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e41c91-404b-43e7-8526-0fc6f70cbe91 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.701984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.702612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.702890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.703161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.703824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.705856 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.820901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1229/2492] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:45 2022] POST /identity/v3/users => generated 304 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:45.826149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.836632 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b43a1-fed7-495d-a739-4764e75e2bb8 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.841370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.842104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.842388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.842697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.843352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.845005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.846521 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:45.858025 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1264/2493] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:45.862218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.877330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a071-018e-45e9-8834-b08f28c0b365 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.882313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af01a9b-6ab6-4c32-9087-78fafee4fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.883125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af01a9b-6ab6-4c32-9087-78fafee4fdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.883594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af01a9b-6ab6-4c32-9087-78fafee4fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.884568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af01a9b-6ab6-4c32-9087-78fafee4fdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.885292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af01a9b-6ab6-4c32-9087-78fafee4fdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.887049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af01a9b-6ab6-4c32-9087-78fafee4fdb5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:45.898646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1230/2494] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:45.903151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:45.913498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081513f7-6630-4951-b2b8-2d2e1948f372 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:45.918165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:45.918873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:45.919145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:45.919417 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] PATH_INFO: `/v3/projects/7a7fafc8f2c644edba0edbe5af360cdb/users/a76b744516cb4691bd9066e03d2b37c6/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:45.936343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7fafc8f2c644edba0edbe5af360cdb, user_id=a76b744516cb4691bd9066e03d2b37c6, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:45.939053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.028587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1265/2495] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:45 2022] PUT /identity/v3/projects/7a7fafc8f2c644edba0edbe5af360cdb/users/a76b744516cb4691bd9066e03d2b37c6/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:17:46.039977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.040259 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.040531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.073752 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] MFA Rules not processed for user `a76b744516cb4691bd9066e03d2b37c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:46.074810 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:46.144394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1231/2496] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:46 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:17:46.149229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:46.216577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc8c26-41fe-4c9a-8a60-0bf49f061c21 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:46.223172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:46.223172 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.223381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.223474 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.224434 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:46.226297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.323160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1266/2497] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:17:46 2022] POST /identity/v3/projects => generated 409 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:46.324880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:46.342319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a03b8-e054-4ffd-9fd6-e96c14f21f7e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:46.355608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:46.436561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:46.437876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.438370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.439245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.440102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:46.445152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.563536 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1232/2498] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:17:46 2022] POST /identity/v3/users => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:17:46.567680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:46.579288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f1c2d-812e-4316-a6cf-f16277ad6867 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:46.585735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:46.587585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.587585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.587585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.587895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:46.589505 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.601589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1267/2499] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:46.606037 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:46.620539 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ced0d-e04c-4d64-87d5-c56fd414c0e0 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:46.627056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:46.628267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.628267 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.628500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] PATH_INFO: `/v3/projects/927cf64f417a4f39a24f00ef035e69e2/users/14c1aa9490614c7a876d9a8d3916421e/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.646648 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927cf64f417a4f39a24f00ef035e69e2, user_id=14c1aa9490614c7a876d9a8d3916421e, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:46.649794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.717766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1233/2500] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:46 2022] PUT /identity/v3/projects/927cf64f417a4f39a24f00ef035e69e2/users/14c1aa9490614c7a876d9a8d3916421e/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:46.721846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:46.732313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcee11e-6bcc-4b86-b19e-d2950066310b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:46.803215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ace95b-631c-4053-9042-000dab753792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:46.803838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ace95b-631c-4053-9042-000dab753792 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.804108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ace95b-631c-4053-9042-000dab753792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.804376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ace95b-631c-4053-9042-000dab753792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.805180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ace95b-631c-4053-9042-000dab753792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:46.807490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ace95b-631c-4053-9042-000dab753792 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.823054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1268/2501] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:17:46.823751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:46.836160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919e54-933c-4580-9a25-b18b4a1f0465 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:46.843602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:46.844298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.845473 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.845920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] PATH_INFO: `/v3/projects/927cf64f417a4f39a24f00ef035e69e2/users/14c1aa9490614c7a876d9a8d3916421e/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.848957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927cf64f417a4f39a24f00ef035e69e2, user_id=14c1aa9490614c7a876d9a8d3916421e, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:46.852868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:46.873115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1234/2502] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:17:46 2022] PUT /identity/v3/projects/927cf64f417a4f39a24f00ef035e69e2/users/14c1aa9490614c7a876d9a8d3916421e/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:46.884842 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:46.885229 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:46.885620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:46.918133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] MFA Rules not processed for user `14c1aa9490614c7a876d9a8d3916421e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:46.919466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:46.973156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:47.032887 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1269/2503] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:46 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:47.044214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:47.046611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:47.107088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ddab-e0d1-489e-969a-ac81f1b798e9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:47.111321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc47f8-8f53-4da5-b3fb-ca189f2524c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:47.111928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc47f8-8f53-4da5-b3fb-ca189f2524c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:47.112196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc47f8-8f53-4da5-b3fb-ca189f2524c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:47.112469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc47f8-8f53-4da5-b3fb-ca189f2524c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:47.123391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc47f8-8f53-4da5-b3fb-ca189f2524c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:47.125362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc47f8-8f53-4da5-b3fb-ca189f2524c7 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:47.138216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1235/2504] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:47 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:47.567321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:47.577708 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f767c-d14c-4449-9815-b68e5f1966ee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:47.582524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:47.583166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:47.583498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:47.594011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:47.669181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:47.671532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:47.685654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1270/2505] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:47 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:17:49.203629 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:49.204110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:49.204232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:49.228266 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:49.229538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:49.297196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1236/2506] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:17:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:49.325535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:49.327119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:49.425072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3422db3-1114-474e-93ac-c8637ae20a43 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:49.430616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65641f1-c763-4aaa-9e22-fa670847ad3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:49.431226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65641f1-c763-4aaa-9e22-fa670847ad3c service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:49.432266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65641f1-c763-4aaa-9e22-fa670847ad3c service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:49.432266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65641f1-c763-4aaa-9e22-fa670847ad3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:49.458333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65641f1-c763-4aaa-9e22-fa670847ad3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:49.458333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65641f1-c763-4aaa-9e22-fa670847ad3c service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:49.466796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1271/2507] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:17:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:17:59.026874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:59.051217 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:59.099085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c07e2-8e80-4ad6-a679-1529b4275f0a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:59.103753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:59.104392 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:59.104665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:59.104935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] PATH_INFO: `/v3/users/a76b744516cb4691bd9066e03d2b37c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:59.106063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76b744516cb4691bd9066e03d2b37c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:59.108040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:59.302958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76b744516cb4691bd9066e03d2b37c6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:59.302958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76b744516cb4691bd9066e03d2b37c6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:59.302958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76b744516cb4691bd9066e03d2b37c6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:59.306808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76b744516cb4691bd9066e03d2b37c6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:59.321559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76b744516cb4691bd9066e03d2b37c6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:17:59.399555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1237/2508] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:17:59 2022] DELETE /identity/v3/users/a76b744516cb4691bd9066e03d2b37c6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:17:59.404439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:59.404963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:59.404963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:59.435252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:17:59.439512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:59.518836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:17:59.531098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1272/2509] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:17:59.542791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:17:59.544263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:17:59.603475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bf171-1b0a-4e7a-9786-1c63eb0dfe60 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:17:59.607369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:17:59.608153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:17:59.608247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:17:59.608527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:17:59.622523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:17:59.625078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:17:59.636430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1238/2510] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:00.170399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:00.176407 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:00.227343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1410750-fb3d-47ba-9040-8a83f88e8dc0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:00.235249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:00.236983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:00.237210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:00.238513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] PATH_INFO: `/v3/projects/7a7fafc8f2c644edba0edbe5af360cdb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:00.241396 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7fafc8f2c644edba0edbe5af360cdb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:00.249232 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:00.368374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7fafc8f2c644edba0edbe5af360cdb', 'request_id': 'req-64ef734c-c8b6-4154-9acf-b34603704360'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.461043 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7fafc8f2c644edba0edbe5af360cdb', 'request_id': 'req-64ef734c-c8b6-4154-9acf-b34603704360'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.468237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7fafc8f2c644edba0edbe5af360cdb', 'request_id': 'req-64ef734c-c8b6-4154-9acf-b34603704360'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.481009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] The token cache is being invalidate because project 7a7fafc8f2c644edba0edbe5af360cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:18:00.481315 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.482087 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.483596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1273/2511] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:18:00 2022] DELETE /identity/v3/projects/7a7fafc8f2c644edba0edbe5af360cdb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:00.487715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:00.499563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181060-3c8c-41ba-b6ff-eb545623716e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:00.542531 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:00.561501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:00.562128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:00.562413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:00.562722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] PATH_INFO: `/v3/users/14c1aa9490614c7a876d9a8d3916421e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:00.563875 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c1aa9490614c7a876d9a8d3916421e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:00.565794 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:00.800622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c1aa9490614c7a876d9a8d3916421e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.801272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c1aa9490614c7a876d9a8d3916421e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.801873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c1aa9490614c7a876d9a8d3916421e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.810950 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c1aa9490614c7a876d9a8d3916421e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.829432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a103384-9378-4c42-810b-bbda5dbfc579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c1aa9490614c7a876d9a8d3916421e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:00.922407 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1239/2512] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:18:00 2022] DELETE /identity/v3/users/14c1aa9490614c7a876d9a8d3916421e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:01.548053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:01.549659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:01.564507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef734c-c8b6-4154-9acf-b34603704360 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:01.632093 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:01.633055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:01.633338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:01.633616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] PATH_INFO: `/v3/projects/927cf64f417a4f39a24f00ef035e69e2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:01.634807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927cf64f417a4f39a24f00ef035e69e2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:01.636761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:01.756703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927cf64f417a4f39a24f00ef035e69e2', 'request_id': 'req-89842f77-ba15-4149-a6db-a3c1bb97caf4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:01.832400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927cf64f417a4f39a24f00ef035e69e2', 'request_id': 'req-89842f77-ba15-4149-a6db-a3c1bb97caf4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:01.839610 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927cf64f417a4f39a24f00ef035e69e2', 'request_id': 'req-89842f77-ba15-4149-a6db-a3c1bb97caf4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:01.849939 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] The token cache is being invalidate because project 927cf64f417a4f39a24f00ef035e69e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:18:01.850222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:01.850862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-89842f77-ba15-4149-a6db-a3c1bb97caf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:01.852578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1274/2513] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:18:01 2022] DELETE /identity/v3/projects/927cf64f417a4f39a24f00ef035e69e2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:01.857542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:01.858021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:01.858115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:01.878108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:01.879357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:01.950519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1240/2514] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:01.959722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:01.960095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:01.960437 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:01.982082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:01.983422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:01.991718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1275/2515] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:01.996295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.006781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a600a-eab3-440e-b5bb-74d9276b25ac None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.011955 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.012581 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.016896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.016896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.016896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:18:02.016896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.016896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.031398 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1241/2516] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:18:02.035735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.040186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:02.045214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8af2a1-c8e2-4698-bee8-3f219e65a342 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.050142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.050886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.051166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.051567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.052200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.054151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.153791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1276/2517] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:02.158287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.170172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacde36-3c60-4264-889a-9ff330114ea5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.223697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:02.240600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.241283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.241566 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.241876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.243084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.245006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.353830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1242/2518] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:18:02 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:02.359998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.379365 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a114dc9-2a4f-4860-9740-2dac80002218 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.384588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.385270 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.385546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.385815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.386473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.388167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.400779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1277/2519] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:18:02.432033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.447806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6de856-cefe-4ea9-b828-439f3885778c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.454021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036411c-923c-45a9-93d5-0db4915ab882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.454668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036411c-923c-45a9-93d5-0db4915ab882 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.454943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036411c-923c-45a9-93d5-0db4915ab882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.455366 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036411c-923c-45a9-93d5-0db4915ab882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.456158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036411c-923c-45a9-93d5-0db4915ab882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.458633 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036411c-923c-45a9-93d5-0db4915ab882 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.472495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1243/2520] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:18:02.477517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.490208 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dab70-2139-4ddb-a992-400ba0379fba admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.496348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.497027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.497359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.497647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] PATH_INFO: `/v3/projects/7e249e6872c846beaefe3f9b760b6319/users/ddd54dd8bc3b41f3b649c2a26e5b54fe/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.516538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e249e6872c846beaefe3f9b760b6319, user_id=ddd54dd8bc3b41f3b649c2a26e5b54fe, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.519369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.603563 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1278/2521] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:18:02 2022] PUT /identity/v3/projects/7e249e6872c846beaefe3f9b760b6319/users/ddd54dd8bc3b41f3b649c2a26e5b54fe/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:02.606402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.607650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.608106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.630842 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:02.633069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:02.718784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1244/2522] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:02.725570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.729008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:02.729754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:02.732429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:02.744367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0abe07-cb8f-4f11-a18e-8db656c8e7d8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.768253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.768892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.769220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.769609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] PATH_INFO: `/v3/users/09483da8f7094da2b51f6d246cec7e46` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.770855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09483da8f7094da2b51f6d246cec7e46)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.773048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.811779 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6097745-64e1-4a52-afca-32492fbd1d4d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:02.816637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8baf91-d87f-4571-a6a2-18e03365209e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:02.817514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8baf91-d87f-4571-a6a2-18e03365209e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:02.817795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8baf91-d87f-4571-a6a2-18e03365209e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:02.818063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8baf91-d87f-4571-a6a2-18e03365209e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:02.832688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8baf91-d87f-4571-a6a2-18e03365209e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:02.834928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8baf91-d87f-4571-a6a2-18e03365209e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:02.850726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1245/2523] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:02.980591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09483da8f7094da2b51f6d246cec7e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:02.981347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09483da8f7094da2b51f6d246cec7e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:02.982133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09483da8f7094da2b51f6d246cec7e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:02.991648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09483da8f7094da2b51f6d246cec7e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:03.017187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09483da8f7094da2b51f6d246cec7e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:03.154266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1279/2524] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:18:02 2022] DELETE /identity/v3/users/09483da8f7094da2b51f6d246cec7e46 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:03.156919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:03.157609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:03.157609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:03.180990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:03.182292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:03.292662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1246/2525] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:03.303134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:03.305796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:03.387063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec313e65-5bf1-4e9f-aad1-76dcd04b9b53 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:03.390907 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97988a-7cce-4dfd-a38c-c8396e5899ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:03.391527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97988a-7cce-4dfd-a38c-c8396e5899ca service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:03.391805 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97988a-7cce-4dfd-a38c-c8396e5899ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:03.392089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97988a-7cce-4dfd-a38c-c8396e5899ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:03.405570 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97988a-7cce-4dfd-a38c-c8396e5899ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:03.407193 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97988a-7cce-4dfd-a38c-c8396e5899ca service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:03.419253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1280/2526] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:04.107591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:04.109702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:04.136434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a17ea9-acb6-4549-abe6-29eb7cbc3019 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:04.148387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:04.148387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:04.148387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:04.148387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] PATH_INFO: `/v3/projects/e6712212ae18438abf5d4c5cf8f16d00` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:04.148735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6712212ae18438abf5d4c5cf8f16d00)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:04.149195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:04.152329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:04.276553 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6712212ae18438abf5d4c5cf8f16d00', 'request_id': 'req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:04.284361 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6712212ae18438abf5d4c5cf8f16d00', 'request_id': 'req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:04.290952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6712212ae18438abf5d4c5cf8f16d00', 'request_id': 'req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:04.412297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] The token cache is being invalidate because project e6712212ae18438abf5d4c5cf8f16d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:18:04.412885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:04.415384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c8ddcdc3-ed12-4255-b0c3-f2c3f2649fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:04.417175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1247/2527] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:18:04 2022] DELETE /identity/v3/projects/e6712212ae18438abf5d4c5cf8f16d00 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:04.422134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:04.422728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:04.422879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:04.466999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:04.466999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:04.508101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:04.630594 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1281/2528] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:18:04.638268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff788205-e560-4878-8545-53457a39916d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:04.638737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff788205-e560-4878-8545-53457a39916d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:04.639085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff788205-e560-4878-8545-53457a39916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:04.660340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ff788205-e560-4878-8545-53457a39916d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:04.661554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ff788205-e560-4878-8545-53457a39916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:04.674355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1248/2529] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:04.679789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:04.690656 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d0720-3bb3-46ff-93cc-ee8723a61813 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:04.695820 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:04.697464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:04.697464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:04.697464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:04.697727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:18:04.697937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:04.699719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:04.714586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1282/2530] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:18:04.717569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff788205-e560-4878-8545-53457a39916d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:04.727969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff788205-e560-4878-8545-53457a39916d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:04.732872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:04.733477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:04.733787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:04.734139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:04.734808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:04.736724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:04.854588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1249/2531] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:18:04 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:04.863662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:04.886649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3b70f-5e99-463f-b20c-58deb0baca21 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:04.982252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:04.982935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:04.983187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:04.983459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:04.984105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:04.986576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:05.118596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:05.118799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1283/2532] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:18:04 2022] POST /identity/v3/users => generated 295 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:05.132996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c155282-4ef4-4b68-a31e-9121f403c676 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:05.150369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:05.151047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:05.151327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:05.151607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:05.152265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:05.154510 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:05.174297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1250/2533] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:18:05.178018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:05.196891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9afa3e-3282-4688-ae6e-7bd964e69213 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:05.207196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cb883-b194-4ac8-8771-547323cd93c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:05.208074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb883-b194-4ac8-8771-547323cd93c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:05.208357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb883-b194-4ac8-8771-547323cd93c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:05.209111 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb883-b194-4ac8-8771-547323cd93c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:05.210205 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7cb883-b194-4ac8-8771-547323cd93c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:05.213264 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7cb883-b194-4ac8-8771-547323cd93c1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:05.234420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1284/2534] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:18:05.241015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:05.253226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce29ab-af96-4fb0-af25-120e9a53dd4c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:05.260080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:05.261761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:05.262256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:05.262772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] PATH_INFO: `/v3/projects/9aca6575ea5f40f2a3be210381a4693a/users/40c5fff1bb314548bd72c3061a0f5468/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:05.293816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aca6575ea5f40f2a3be210381a4693a, user_id=40c5fff1bb314548bd72c3061a0f5468, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:05.297878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:05.385816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1251/2535] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:18:05 2022] PUT /identity/v3/projects/9aca6575ea5f40f2a3be210381a4693a/users/40c5fff1bb314548bd72c3061a0f5468/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:05.399894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ae6cc-5071-49b5-845d-89adec39c364 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:05.400468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ae6cc-5071-49b5-845d-89adec39c364 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:05.401807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ae6cc-5071-49b5-845d-89adec39c364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:05.467660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-763ae6cc-5071-49b5-845d-89adec39c364 None None] MFA Rules not processed for user `40c5fff1bb314548bd72c3061a0f5468`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:05.468990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-763ae6cc-5071-49b5-845d-89adec39c364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:05.642597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1285/2536] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:05.663591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:05.667372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:05.768923 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822c2d9-5f6a-4b44-ac81-2a779d20cb94 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:05.776218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:05.777538 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:05.778062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:05.778805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:05.795218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:05.796713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:05.809734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1252/2537] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:06.683791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d9957-c96c-4477-aeaf-7c03e21e367d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:06.684317 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d9957-c96c-4477-aeaf-7c03e21e367d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:06.684550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d9957-c96c-4477-aeaf-7c03e21e367d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:06.725023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6e9d9957-c96c-4477-aeaf-7c03e21e367d None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:06.725023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e9d9957-c96c-4477-aeaf-7c03e21e367d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:06.745891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:06.854001 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1286/2538] 10.222.0.46 () {58 vars in 967 bytes} [Wed Sep 7 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:06.873459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:06.875848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:06.958772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:07.012053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a8ea4-4d98-4f9c-8258-112e44ba4c72 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:07.018753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:07.019383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:07.019795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:07.020165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:07.029822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:07.031798 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:07.041735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1253/2539] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:18:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:18:10.122925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.123455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.123455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.153572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] MFA Rules not processed for user `ddd54dd8bc3b41f3b649c2a26e5b54fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:10.154874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:10.230456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1287/2540] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:10.235952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:10.239525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:10.310527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a63cce-ac1d-49d5-94ec-d4929893ccf0 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:10.315560 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:10.317896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.317896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.317896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.317896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:10.319360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:10.399090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1254/2541] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:18:10 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:10.403250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:10.415258 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df7824-dbbe-44ff-bed7-050ef361fa82 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:10.475539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:10.476863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.477143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.477416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.478062 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:10.480223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:10.573681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1288/2542] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:18:10 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:10.577649 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:10.588612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2df785-1ae7-42ad-ae65-44e13de1c744 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:10.593876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:10.594472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.594781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.595056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.595699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:10.597300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:10.608766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1255/2543] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:18:10.612897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:10.622964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c25cf-15e9-4349-8e36-c26d3e505726 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:10.627516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:10.631484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.631484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.631484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] PATH_INFO: `/v3/projects/fc9b2950d72c414fb68c055b15deba85/users/251d37e79f5143deaa37f45a52012aa6/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.644340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b2950d72c414fb68c055b15deba85, user_id=251d37e79f5143deaa37f45a52012aa6, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:10.647401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:10.711579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1289/2544] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:18:10 2022] PUT /identity/v3/projects/fc9b2950d72c414fb68c055b15deba85/users/251d37e79f5143deaa37f45a52012aa6/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:10.715612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:10.726487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e635457-0b63-4b78-85a4-60dce7152338 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:10.793001 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339dc7e-7495-4930-8cc5-3df10b3c95bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:10.793622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339dc7e-7495-4930-8cc5-3df10b3c95bd admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.793898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339dc7e-7495-4930-8cc5-3df10b3c95bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.794177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339dc7e-7495-4930-8cc5-3df10b3c95bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.794863 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339dc7e-7495-4930-8cc5-3df10b3c95bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:10.796642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339dc7e-7495-4930-8cc5-3df10b3c95bd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:10.812713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1256/2545] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:18:10.812817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:10.823916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f564a0-1542-4a3a-9eb3-34a2fc249fdc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:10.828917 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:10.829845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:10.830141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:10.830499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] PATH_INFO: `/v3/projects/fc9b2950d72c414fb68c055b15deba85/users/251d37e79f5143deaa37f45a52012aa6/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:10.832553 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9b2950d72c414fb68c055b15deba85, user_id=251d37e79f5143deaa37f45a52012aa6, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:10.835890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:10.846591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1290/2546] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:18:10 2022] PUT /identity/v3/projects/fc9b2950d72c414fb68c055b15deba85/users/251d37e79f5143deaa37f45a52012aa6/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:18.031691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c505cc2-d434-42de-b836-99e26d6878ab None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:18.032307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c505cc2-d434-42de-b836-99e26d6878ab None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:18.032307 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c505cc2-d434-42de-b836-99e26d6878ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:18.067170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8c505cc2-d434-42de-b836-99e26d6878ab None None] MFA Rules not processed for user `251d37e79f5143deaa37f45a52012aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:18.067170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8c505cc2-d434-42de-b836-99e26d6878ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:18.087948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:18.150166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1257/2547] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:18.158108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:18.159845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:18.166109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:18.257886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d77b-7866-4d87-b94f-cbca113b3ab4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:18.262361 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f6d9e85ceed45e186421315737f9e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:18.267240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:18.267240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:18.267240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:18.359642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:18.361098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:18.382630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1291/2548] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:18.678503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6c011-f5ba-45c6-992a-39ade382297f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:18.679090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6c011-f5ba-45c6-992a-39ade382297f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:18.679568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6c011-f5ba-45c6-992a-39ade382297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:18.739490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e1e6c011-f5ba-45c6-992a-39ade382297f None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:18.741023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e1e6c011-f5ba-45c6-992a-39ade382297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:18.814157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1258/2549] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:18.830167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:18.957990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eed5d-6dba-478c-b727-200cdcfbb1cd service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:18.965967 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:18.967299 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:18.967659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:18.968036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:18.979628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:18.982906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:19.000157 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1292/2550] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:18:19.501301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceb575-c2e5-40dc-8ca7-2198df7b100d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:19.501780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceb575-c2e5-40dc-8ca7-2198df7b100d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:19.501878 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceb575-c2e5-40dc-8ca7-2198df7b100d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:19.527888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3fceb575-c2e5-40dc-8ca7-2198df7b100d None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:19.529429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3fceb575-c2e5-40dc-8ca7-2198df7b100d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:19.538298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1259/2551] 10.222.0.46 () {58 vars in 967 bytes} [Wed Sep 7 19:18:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:19.556398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:19.567511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6c992-2866-49d9-a7a0-8552395e5490 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:19.573628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:19.578932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:19.578932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:19.578932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:19.584362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:19.586849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:19.595747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1293/2552] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:18:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 19:18:19.925516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:19.925881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:19.926229 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:19.961827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] MFA Rules not processed for user `f9db5f73350c460f99e4ab5cdbe6907a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:19.963814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:20.036802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1260/2553] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:24.319906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:24.321962 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:24.389880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:24.447966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aa245-f04b-492c-92cd-92040d4b7a44 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:24.453233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:24.455166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:24.455166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:24.455166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] PATH_INFO: `/v3/users/54bff8714b5b4f0d96b8fbeda69e4e98` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:24.456117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bff8714b5b4f0d96b8fbeda69e4e98)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:24.458171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:24.688221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bff8714b5b4f0d96b8fbeda69e4e98'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:24.691248 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bff8714b5b4f0d96b8fbeda69e4e98'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:24.693891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bff8714b5b4f0d96b8fbeda69e4e98'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:24.706398 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bff8714b5b4f0d96b8fbeda69e4e98'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:24.723912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e61f1113-5728-496a-87b6-c00be8ab5264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bff8714b5b4f0d96b8fbeda69e4e98'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:24.850150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1294/2554] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:18:24 2022] DELETE /identity/v3/users/54bff8714b5b4f0d96b8fbeda69e4e98 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:25.348992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:25.363500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d53d6-6c8c-405b-8d61-1720d95f8f11 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:25.455290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:25.455926 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:25.456200 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:25.456469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] PATH_INFO: `/v3/projects/702737b15c8e484d8aba627563179edf` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:25.457591 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702737b15c8e484d8aba627563179edf)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:25.459748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:25.470100 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:25.601420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702737b15c8e484d8aba627563179edf', 'request_id': 'req-174547a0-2035-44ef-a045-607d217f0669'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:25.610144 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702737b15c8e484d8aba627563179edf', 'request_id': 'req-174547a0-2035-44ef-a045-607d217f0669'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:25.617590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702737b15c8e484d8aba627563179edf', 'request_id': 'req-174547a0-2035-44ef-a045-607d217f0669'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:25.717904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] The token cache is being invalidate because project 702737b15c8e484d8aba627563179edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:18:25.718204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:25.718905 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-174547a0-2035-44ef-a045-607d217f0669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:25.720464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1261/2555] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:18:25 2022] DELETE /identity/v3/projects/702737b15c8e484d8aba627563179edf => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:25.725596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:25.726651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:25.726761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:25.758204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:25.761958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:25.835916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1295/2556] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:25.910843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:25.911153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:25.911430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:25.948096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:25.948096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:25.968784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:25.975781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1262/2557] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:25.980261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cc392-2154-4911-83ba-00db834ddb22 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:25.988944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:25.989749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:25.990136 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:25.990595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:25.991409 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:18:25.991825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:25.993573 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:26.016942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:26.023841 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1296/2558] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:18:26.027846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733ab1-f808-47ab-bff0-a4e8f764c235 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:26.033569 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:26.034295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:26.034608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:26.034882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:26.035517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:26.037503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:26.198277 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1263/2559] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:18:26 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:18:26.202639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:26.225566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062c47-9410-459f-82f0-9dc0782d0124 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:26.292649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:26.322663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:26.323568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:26.323851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:26.324176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:26.324793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:26.326901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:26.570845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1297/2560] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:18:26 2022] POST /identity/v3/users => generated 293 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:26.575373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:26.586026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f5b51-ee20-496c-8b50-8fa3e0c4ea74 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:26.590625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:26.595537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:26.595537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:26.595537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:26.595537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:26.595537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:26.607208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1264/2561] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:18:26.608423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:26.619222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea569f69-344b-4141-bfcf-98506d219885 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:26.625191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d833877-b644-4000-b909-adf58be2c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:26.625807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d833877-b644-4000-b909-adf58be2c930 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:26.626084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d833877-b644-4000-b909-adf58be2c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:26.626354 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d833877-b644-4000-b909-adf58be2c930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:26.627030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d833877-b644-4000-b909-adf58be2c930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:26.630307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d833877-b644-4000-b909-adf58be2c930 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:26.640469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1298/2562] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:18:26.644329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:26.655975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a800-85c3-4cd3-abdd-41536eac829a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:26.663978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:26.664840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:26.665391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:26.665766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] PATH_INFO: `/v3/projects/b70cded0fc074bd0bc3f6e853facb4b2/users/fb6fb134879849358b36143ee8e2943e/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:26.686391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70cded0fc074bd0bc3f6e853facb4b2, user_id=fb6fb134879849358b36143ee8e2943e, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:26.689335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:26.752262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1265/2563] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:18:26 2022] PUT /identity/v3/projects/b70cded0fc074bd0bc3f6e853facb4b2/users/fb6fb134879849358b36143ee8e2943e/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:26.762983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:26.763848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:26.764413 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:26.815249 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] MFA Rules not processed for user `fb6fb134879849358b36143ee8e2943e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:26.816768 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:26.979015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:26.981028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:26.986638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1299/2564] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:27.089562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d6a5d-f2e0-4c87-8e75-e2b35ae4c330 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:27.099443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdefd0-995e-44e3-b1f7-14001fb11ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:27.100102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdefd0-995e-44e3-b1f7-14001fb11ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:27.100391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdefd0-995e-44e3-b1f7-14001fb11ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:27.101216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdefd0-995e-44e3-b1f7-14001fb11ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:27.111494 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdefd0-995e-44e3-b1f7-14001fb11ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:27.112938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdefd0-995e-44e3-b1f7-14001fb11ba4 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:27.130866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1266/2565] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:29.407387 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:29.409269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:29.525321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40be61d-0d9f-4c1a-88c0-2b0e6bfe16bf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:29.530534 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:29.531207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:29.531485 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:29.531759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] PATH_INFO: `/v3/users/40c5fff1bb314548bd72c3061a0f5468` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:29.532930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c5fff1bb314548bd72c3061a0f5468)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:29.536996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:29.774769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c5fff1bb314548bd72c3061a0f5468'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:29.776055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c5fff1bb314548bd72c3061a0f5468'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:29.777083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c5fff1bb314548bd72c3061a0f5468'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:29.779306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:29.785344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c5fff1bb314548bd72c3061a0f5468'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:29.804339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c5fff1bb314548bd72c3061a0f5468'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:29.898333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1300/2566] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:18:29 2022] DELETE /identity/v3/users/40c5fff1bb314548bd72c3061a0f5468 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:29.903589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:29.903589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:29.903589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:29.922215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:29.923521 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:29.951109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:30.001919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1267/2567] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:18:30.014097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:30.015611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:30.091902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bf4c7-a27d-4a81-8370-0f02e444d08e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:30.095639 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:30.096298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:30.096601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:30.096879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:30.107649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:30.232577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:30.232577 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1301/2568] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:30.496676 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:30.498113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:30.522845 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711377a-3962-4cf3-9d06-316805ea7bad None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:30.529952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:30.530623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:30.530904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:30.531177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] PATH_INFO: `/v3/users/fb6fb134879849358b36143ee8e2943e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:30.534632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6fb134879849358b36143ee8e2943e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:30.536567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:30.748412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:30.750910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:30.793041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d504eb-ab08-4ce6-b794-ab0beee9243a service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:30.794579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6fb134879849358b36143ee8e2943e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:30.796146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6fb134879849358b36143ee8e2943e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:30.796612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6fb134879849358b36143ee8e2943e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:30.800964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:30.801683 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:30.801872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:30.802149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] PATH_INFO: `/v3/projects/9aca6575ea5f40f2a3be210381a4693a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:30.807794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aca6575ea5f40f2a3be210381a4693a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:30.812121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:30.814908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6fb134879849358b36143ee8e2943e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:30.837443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a645780d-77f4-4f0b-b77d-b9b68ac460f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6fb134879849358b36143ee8e2943e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:30.986290 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1268/2569] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:18:30 2022] DELETE /identity/v3/users/fb6fb134879849358b36143ee8e2943e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:30.990820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:30.991117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:30.991393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:31.010146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:31.011378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:31.055195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aca6575ea5f40f2a3be210381a4693a', 'request_id': 'req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.077612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1269/2570] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:18:31.088054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:31.095272 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:31.134205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:31.145462 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a85112-8e09-4d7d-9256-cbc61cf33eb3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:31.193110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:31.193718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:31.194000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:31.194828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:31.241482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aca6575ea5f40f2a3be210381a4693a', 'request_id': 'req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.254663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aca6575ea5f40f2a3be210381a4693a', 'request_id': 'req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.269458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] The token cache is being invalidate because project 9aca6575ea5f40f2a3be210381a4693a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:18:31.269867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.270852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8c8b159-2ec1-4d08-a722-d7ac43e1b6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.273593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1302/2571] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:18:30 2022] DELETE /identity/v3/projects/9aca6575ea5f40f2a3be210381a4693a => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:18:31.281773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:31.284323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:31.297596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1270/2572] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:18:31.615147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26c83-2f28-4c7c-b5f1-24a374c3fe98 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:31.615290 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26c83-2f28-4c7c-b5f1-24a374c3fe98 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:31.615635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26c83-2f28-4c7c-b5f1-24a374c3fe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:31.637924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e8e26c83-2f28-4c7c-b5f1-24a374c3fe98 None None] MFA Rules not processed for user `ddd54dd8bc3b41f3b649c2a26e5b54fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:31.639720 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8e26c83-2f28-4c7c-b5f1-24a374c3fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:31.711886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:31.720894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1303/2573] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:31.783971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:31.783971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:31.802813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89436922-06a3-49f2-ba55-fd90f64f9c90 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:31.817027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:31.817027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:31.817027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:31.817027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] PATH_INFO: `/v3/projects/b70cded0fc074bd0bc3f6e853facb4b2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:31.817759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70cded0fc074bd0bc3f6e853facb4b2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:31.818154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:31.960574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70cded0fc074bd0bc3f6e853facb4b2', 'request_id': 'req-a2339788-646c-41dc-a82c-4d8b452bc251'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.967459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70cded0fc074bd0bc3f6e853facb4b2', 'request_id': 'req-a2339788-646c-41dc-a82c-4d8b452bc251'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:31.973211 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70cded0fc074bd0bc3f6e853facb4b2', 'request_id': 'req-a2339788-646c-41dc-a82c-4d8b452bc251'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:32.081513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] The token cache is being invalidate because project b70cded0fc074bd0bc3f6e853facb4b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:18:32.082188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:32.082702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a2339788-646c-41dc-a82c-4d8b452bc251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:18:32.084409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1271/2574] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:18:31 2022] DELETE /identity/v3/projects/b70cded0fc074bd0bc3f6e853facb4b2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:18:32.089731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.090201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.090304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.110310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:32.111571 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:32.174500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1304/2575] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:32.182059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.182351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.182733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.203380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:32.204616 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:32.215848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1272/2576] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:32.220250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.230240 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29ba15-114a-4320-b7ba-4cee3d689188 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.235713 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.236731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.236731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.236919 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.237498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:18:32.237803 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.239452 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.251872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1305/2577] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:18:32.256280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.266213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dce1fb-6b82-40fd-80aa-240bc5371416 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.271179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.272042 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.272313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.273173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.273793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.275762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.373646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1273/2578] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:18:32 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:18:32.374928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.385697 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0421e4-0b72-4312-8f8e-59a9ba7cc0df admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.443040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.443676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.443930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.444233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.444870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.446815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.535822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1306/2579] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:18:32 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:18:32.536027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.543155 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:32.576453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f07083-3f65-4dfa-abd4-2ebacfbf2c08 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.592094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.592710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.592981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.593246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.593895 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.595664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.609321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1274/2580] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:18:32.610597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.620858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e30074-4a0a-4838-980c-6303c609ad49 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.626514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c9d28-30d6-4bdb-b5f8-94823e698343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.627104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c9d28-30d6-4bdb-b5f8-94823e698343 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.627377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c9d28-30d6-4bdb-b5f8-94823e698343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.627645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c9d28-30d6-4bdb-b5f8-94823e698343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.628285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c9d28-30d6-4bdb-b5f8-94823e698343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.629856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c9d28-30d6-4bdb-b5f8-94823e698343 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.649058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.649517 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1307/2581] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:18:32.660327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819209e-93c0-4065-8de0-b9da5a911440 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.665559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.667418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.667418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.667418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] PATH_INFO: `/v3/projects/ae71e430b86c435893eb2617d1fa5e33/users/300ae5cd8ead404c8d0b425e635b04b7/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.682220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae71e430b86c435893eb2617d1fa5e33, user_id=300ae5cd8ead404c8d0b425e635b04b7, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.684943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.755928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1275/2582] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:18:32 2022] PUT /identity/v3/projects/ae71e430b86c435893eb2617d1fa5e33/users/300ae5cd8ead404c8d0b425e635b04b7/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:18:32.767856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99bb8-4a2a-4e4c-b423-10abc75d008c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.768138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99bb8-4a2a-4e4c-b423-10abc75d008c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.768412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99bb8-4a2a-4e4c-b423-10abc75d008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.797595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9ee99bb8-4a2a-4e4c-b423-10abc75d008c None None] MFA Rules not processed for user `300ae5cd8ead404c8d0b425e635b04b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:32.798817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9ee99bb8-4a2a-4e4c-b423-10abc75d008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:32.851742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:32.861171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1308/2583] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:32.874139 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:32.875942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:32.938776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b91de-eee9-4917-973f-1e045a04b978 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:32.943963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06a9c5-3627-491f-9970-65026a82fe62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:32.943963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a9c5-3627-491f-9970-65026a82fe62 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:32.943963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a9c5-3627-491f-9970-65026a82fe62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:32.944831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a9c5-3627-491f-9970-65026a82fe62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:32.953711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06a9c5-3627-491f-9970-65026a82fe62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:32.955258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06a9c5-3627-491f-9970-65026a82fe62 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:32.965257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1276/2584] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:18:34.693589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:34.694071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:34.694925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:34.714024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] MFA Rules not processed for user `300ae5cd8ead404c8d0b425e635b04b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:34.715277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:34.723037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1309/2585] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:36.554190 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:36.554704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:36.554948 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:36.577669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:36.578952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:36.645738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1277/2586] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:18:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:18:36.660356 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:36.662522 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:36.740140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c9949-5ed8-48f0-8bbd-bce7274fd0e2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:36.746165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ab4d-103f-41d7-9644-508550ad1785 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:36.746824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ab4d-103f-41d7-9644-508550ad1785 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:36.747106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ab4d-103f-41d7-9644-508550ad1785 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:36.747379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ab4d-103f-41d7-9644-508550ad1785 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:36.755944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ab4d-103f-41d7-9644-508550ad1785 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:36.758052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ab4d-103f-41d7-9644-508550ad1785 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:36.766650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1310/2587] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:18:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 19:18:46.637904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:46.763088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=38 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:18:55.538756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:18:55.540208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:55.621178 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62a438-2044-4ead-bb16-8fefd0e247d3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:18:55.626850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:18:55.628043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:55.628043 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:55.628244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:55.630382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:55.712502 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:18:55.714114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37133-6054-40ac-a4b3-31a3aacb18b6 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:18:55.739059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1278/2588] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:18:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:18:56.461178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:18:56.461774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:18:56.461774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:18:56.481885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] MFA Rules not processed for user `847f6bef4e7b44f4b0adf1bda9dadb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:18:56.483217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:18:56.548324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1311/2589] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:18:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:03.862432 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a2d98-899a-4b8f-8b20-07bd5f776967 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:03.862985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a2d98-899a-4b8f-8b20-07bd5f776967 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:03.862985 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a2d98-899a-4b8f-8b20-07bd5f776967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:03.882669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dd6a2d98-899a-4b8f-8b20-07bd5f776967 None None] MFA Rules not processed for user `920bdd998f0c4b62b108afc799a43964`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:19:03.886056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dd6a2d98-899a-4b8f-8b20-07bd5f776967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:03.940159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1279/2590] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:19:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:06.537648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=25 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:11.350894 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:11.353680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:11.365744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07967c66-c66f-4e1d-b1ed-633decedb5fb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:11.369979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:11.370634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:11.370929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:11.371181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:11.373006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:11.442540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:11.444191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:11.458006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1312/2591] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:19:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:19:12.466982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4fafc-9d14-417b-98be-07c40e633e97 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:12.467530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4fafc-9d14-417b-98be-07c40e633e97 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:12.467530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4fafc-9d14-417b-98be-07c40e633e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:12.487056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-59d4fafc-9d14-417b-98be-07c40e633e97 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:19:12.488301 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-59d4fafc-9d14-417b-98be-07c40e633e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:12.495627 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1280/2592] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:12.518028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:12.530942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06be47-1cab-4203-8740-b1f35944482b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:12.536102 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:12.536723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:12.542189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:12.542189 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:12.546400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:12.548029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:12.557190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1313/2593] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:19:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:19:13.412503 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff71a32-b7fc-4380-9163-335879daf6d9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:13.412792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff71a32-b7fc-4380-9163-335879daf6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:13.413069 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff71a32-b7fc-4380-9163-335879daf6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:13.433140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5ff71a32-b7fc-4380-9163-335879daf6d9 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:19:13.434348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5ff71a32-b7fc-4380-9163-335879daf6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:13.441674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1281/2594] 10.222.0.46 () {58 vars in 967 bytes} [Wed Sep 7 19:19:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:13.454457 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:13.465350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e207-e177-41fa-a216-f483162003a8 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:13.470593 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:13.471194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:13.471475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:13.471752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:13.480657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:13.482230 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:13.490582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1314/2595] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:19:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:19:14.204370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e930cd-ca0b-4996-85f5-46085136dd79 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:14.204977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e930cd-ca0b-4996-85f5-46085136dd79 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:14.205433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e930cd-ca0b-4996-85f5-46085136dd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:14.248636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-38e930cd-ca0b-4996-85f5-46085136dd79 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:19:14.250483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-38e930cd-ca0b-4996-85f5-46085136dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:14.274175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:14.284850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1282/2596] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:14.289870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f2693-b780-4c08-b350-6d614c84b403 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:14.296052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:14.297304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:14.297831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:14.298319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:14.310744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:14.316016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:14.328462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1315/2597] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:19:14.934147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:14.934434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:14.934749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:14.955527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:19:14.956838 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:14.965154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1283/2598] 10.222.0.46 () {58 vars in 967 bytes} [Wed Sep 7 19:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:14.977543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:14.988711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f09b2f-ed43-47a7-9cf5-c4bd5578bbaf service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:14.994253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bccf7-9bcc-4b7b-8459-0fe7ba37a994 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:14.994923 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bccf7-9bcc-4b7b-8459-0fe7ba37a994 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:14.995195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bccf7-9bcc-4b7b-8459-0fe7ba37a994 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:14.995465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bccf7-9bcc-4b7b-8459-0fe7ba37a994 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:15.004156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bccf7-9bcc-4b7b-8459-0fe7ba37a994 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:15.005722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bccf7-9bcc-4b7b-8459-0fe7ba37a994 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:15.014875 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1316/2599] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:19:17.930885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:17.937249 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:18.016744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9870d9-f25c-484b-83fc-f209300592d9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:18.021554 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:18.022184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:18.022459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:18.022754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] PATH_INFO: `/v3/users/300ae5cd8ead404c8d0b425e635b04b7` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:18.025374 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] RBAC: Authorizing `identity:delete_user(user_id=300ae5cd8ead404c8d0b425e635b04b7)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:18.027438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:18.207381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300ae5cd8ead404c8d0b425e635b04b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.208171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300ae5cd8ead404c8d0b425e635b04b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.209002 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300ae5cd8ead404c8d0b425e635b04b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.217410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300ae5cd8ead404c8d0b425e635b04b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.230119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300ae5cd8ead404c8d0b425e635b04b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.334639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1284/2600] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:19:17 2022] DELETE /identity/v3/users/300ae5cd8ead404c8d0b425e635b04b7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:19:18.340211 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:18.340799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:18.340799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:18.361063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:19:18.362326 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:18.422000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1317/2601] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:19:18.432159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:18.433612 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:18.493098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d887-7637-49e1-8e1d-5b1a017596be admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:18.496813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:18.497428 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:18.497704 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:18.497975 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:18.509637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:18.511947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:18.522808 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1285/2602] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:19:18.776822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:18.778358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:18.791482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ded8a7-47ff-4e5f-accb-1e94dff65716 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:18.796298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:18.796786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:18.797191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:18.797471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] PATH_INFO: `/v3/projects/ae71e430b86c435893eb2617d1fa5e33` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:18.798941 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae71e430b86c435893eb2617d1fa5e33)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:18.800833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:18.882199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:18.891811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae71e430b86c435893eb2617d1fa5e33', 'request_id': 'req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.934761 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae71e430b86c435893eb2617d1fa5e33', 'request_id': 'req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.941476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae71e430b86c435893eb2617d1fa5e33', 'request_id': 'req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.952129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] The token cache is being invalidate because project ae71e430b86c435893eb2617d1fa5e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:19:18.952399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.953035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:19:18.966187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1318/2603] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:19:18 2022] DELETE /identity/v3/projects/ae71e430b86c435893eb2617d1fa5e33 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:19:28.517974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:29.009304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:40.529197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:40.530748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:40.542367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee34e-a1bc-49db-ac60-c69d39f29ceb service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:40.592705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:40.593308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:40.593589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:40.593864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:40.595719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:40.669609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:40.671255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:40.684890 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1286/2604] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:19:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:19:41.877495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:41.878951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:41.891744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae10e78-bd4e-4c3c-b18f-2efe9142fd60 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:41.895375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:41.895988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:41.896280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:41.896546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:41.898197 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:41.912159 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:41.913545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:41.924400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1319/2605] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:19:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:19:43.281624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:19:43.283062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:43.338540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55590-6e21-4273-8b11-a7904c0e40c8 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:19:43.342650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:19:43.343248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:19:43.343527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:19:43.343799 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:19:43.345504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:19:43.400646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:43.403636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:19:43.405128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:19:43.414320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1287/2606] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:19:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:19:51.919685 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:19:53.426146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:00.042170 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:00.043858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:00.058445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aac307-f539-40ad-bd28-8f9c707cf1ed service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:00.064141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:00.064827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:00.065174 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:00.065513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:00.067360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:00.080554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:00.082179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:00.093219 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1320/2607] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:06.215990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:06.217450 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:06.237423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da873-01f1-4609-afa9-756064c0ff0a service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:06.243170 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:06.243773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:06.244051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:06.244325 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:06.246651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:06.305660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:06.307338 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677e7a7-ee05-475b-96a4-4aace1006c93 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:06.317154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1288/2608] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:06.569152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:06.578931 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31e022-ee3b-4ba8-b32e-071671401c3f service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:06.582638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:06.583247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:06.583504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:06.583769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:06.585567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:06.648513 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:06.650201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff46d9-9728-44fc-ac2b-0e1d559627e4 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:06.660945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1321/2609] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:09.945615 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58007c-eb03-494f-87f1-4b7af37add0b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:09.946110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58007c-eb03-494f-87f1-4b7af37add0b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:09.946239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58007c-eb03-494f-87f1-4b7af37add0b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:09.947594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1289/2610] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:20:09.977733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:09.978252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:09.978355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:09.998733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:09.999956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:10.064888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1322/2611] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:20:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:20:10.076438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58007c-eb03-494f-87f1-4b7af37add0b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:10.078165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bd58007c-eb03-494f-87f1-4b7af37add0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:10.092882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:10.094318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:10.101662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58007c-eb03-494f-87f1-4b7af37add0b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:10.107082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:10.107720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:10.107853 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a818e00-5f25-44c8-bd9a-22bed52375ca None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:10.107995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:10.108447 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:10.112651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:10.122628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:10.122628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:10.122628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] PATH_INFO: `/v3/users/847f6bef4e7b44f4b0adf1bda9dadb7e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:10.122628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=847f6bef4e7b44f4b0adf1bda9dadb7e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:10.122628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:10.123066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:10.123066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:10.128839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1290/2612] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:10.264319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847f6bef4e7b44f4b0adf1bda9dadb7e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.265064 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847f6bef4e7b44f4b0adf1bda9dadb7e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.265865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847f6bef4e7b44f4b0adf1bda9dadb7e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.273190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847f6bef4e7b44f4b0adf1bda9dadb7e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.285897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:10.287242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8f4dd9f-b4bb-4a9c-b9fa-38a3f7c4fb0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847f6bef4e7b44f4b0adf1bda9dadb7e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.342721 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1323/2613] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:20:10 2022] DELETE /identity/v3/users/847f6bef4e7b44f4b0adf1bda9dadb7e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:20:10.597753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:10.609186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c954d-fed1-4dbf-b7b5-4142ba590376 service placement] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:10.668827 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:10.669375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:10.669651 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:10.669925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] PATH_INFO: `/v3/projects/f0ca8c7ab1e64a0bb02a4941fe7b8105` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:10.671062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ca8c7ab1e64a0bb02a4941fe7b8105)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:10.672969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:10.786356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ca8c7ab1e64a0bb02a4941fe7b8105', 'request_id': 'req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.792372 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ca8c7ab1e64a0bb02a4941fe7b8105', 'request_id': 'req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.798734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ca8c7ab1e64a0bb02a4941fe7b8105', 'request_id': 'req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.860993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] The token cache is being invalidate because project f0ca8c7ab1e64a0bb02a4941fe7b8105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:20:10.861222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.861869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0cec3025-3d8d-4352-85bb-e4bb31b3b5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:10.863356 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1291/2614] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:20:10 2022] DELETE /identity/v3/projects/f0ca8c7ab1e64a0bb02a4941fe7b8105 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:20:10.868724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:10.869008 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:10.869287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:10.890692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:10.891893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:10.954620 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1324/2615] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:10.962678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:10.963163 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:10.963268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:10.983337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:10.984550 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:10.992387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1292/2616] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:10.997338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.006940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242d266-821e-4ce3-b8d8-0dd488be29bc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.011390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.011995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.012269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.012535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.013167 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:20:11.013462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.015135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.026640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1325/2617] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:20:11.030894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.040125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60cf69-c065-45a7-b813-df913e02be97 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.045274 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.045859 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.046137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.046409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.047063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.048969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.054138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:11.114320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1293/2618] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:20:11 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:20:11.118748 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.129379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112656fb-19df-41bb-a7d1-c8cb376893c7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.186995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.187506 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.187784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.188055 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.188691 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.190600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.254884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1326/2619] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:20:11 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:20:11.259035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.269870 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518530e-3ead-428b-a201-d5ba36e6669a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.274791 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.275389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.275663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.275932 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.276572 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.278153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.287993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1294/2620] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:20:11.291985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.301904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90351cdb-6b4d-4e2d-90fb-1e2aa8f75cf9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.306459 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc44299-1a18-4f73-a67b-e3b533f2b976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.307108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44299-1a18-4f73-a67b-e3b533f2b976 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.307388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44299-1a18-4f73-a67b-e3b533f2b976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.307661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44299-1a18-4f73-a67b-e3b533f2b976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.308297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44299-1a18-4f73-a67b-e3b533f2b976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.310362 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44299-1a18-4f73-a67b-e3b533f2b976 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.320676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1327/2621] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:20:11.325210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.335030 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f27b0-4816-4545-bcc6-cadaa4911960 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.339573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.340715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.340992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.341261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] PATH_INFO: `/v3/projects/53d22963975b4bb891b1858f4afe3275/users/7c8142614fe149fd9593cfba97ecad18/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.356040 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d22963975b4bb891b1858f4afe3275, user_id=7c8142614fe149fd9593cfba97ecad18, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.358696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.426929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1295/2622] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:20:11 2022] PUT /identity/v3/projects/53d22963975b4bb891b1858f4afe3275/users/7c8142614fe149fd9593cfba97ecad18/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:20:11.432132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fb453-aa82-4f62-b3a9-30b75d4d750c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.432427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fb453-aa82-4f62-b3a9-30b75d4d750c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.432706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fb453-aa82-4f62-b3a9-30b75d4d750c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.452237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-405fb453-aa82-4f62-b3a9-30b75d4d750c None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:11.453420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-405fb453-aa82-4f62-b3a9-30b75d4d750c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:11.504857 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:11.516492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1328/2623] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:11.526481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:11.529054 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:11.585126 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49337a-dc8c-44af-a46c-319e8d9b748b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:11.588766 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:11.589363 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:11.589640 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:11.590568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:11.599644 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:11.601102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:11.611167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1296/2624] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:17.505699 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10b133-dd4a-48ac-b3a7-75b4b6213d53 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:17.506194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10b133-dd4a-48ac-b3a7-75b4b6213d53 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:17.506194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10b133-dd4a-48ac-b3a7-75b4b6213d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:17.541828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3f10b133-dd4a-48ac-b3a7-75b4b6213d53 None None] MFA Rules not processed for user `7c8142614fe149fd9593cfba97ecad18`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:17.543158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f10b133-dd4a-48ac-b3a7-75b4b6213d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:17.622112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1329/2625] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:20:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:17.637094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:17.638660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:17.716795 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be4e7f-f717-43a3-b701-89b77a797297 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:17.722382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:17.723018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:17.723295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:17.723564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:17.731744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:17.733317 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:17.742262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1297/2626] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:20.155681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:20.158281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:20.158861 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:20.181856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] MFA Rules not processed for user `7c8142614fe149fd9593cfba97ecad18`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:20.183950 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:20.198191 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1330/2627] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:20:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:20.522971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:20.533577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4ba7-bbf3-42b1-9af6-a3e094f2c183 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:20.537489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:20.538098 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:20.538373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:20.538680 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:20.540439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:20.550734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:20.552174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab90fe-e1ee-4b9d-b33a-65ee08686815 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:20.560916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1298/2628] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:20.967569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:20.971671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:21.047530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204abf4b-2fdd-4db4-9646-1b255404b03e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:21.052545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:21.053146 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:21.053498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:21.053772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:21.055523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:21.068428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:21.069986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:21.081489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1331/2629] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:22.360655 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:22.361160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:22.361304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:22.382285 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:22.385413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:22.405828 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:22.445700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1299/2630] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:20:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:20:22.602817 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:22.604460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:22.692565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a65a45-1e4a-4baf-ba4d-2e0962e1d140 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:22.697604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:22.701550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:22.701550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:22.701550 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:22.708607 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:22.710307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:22.714117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:22.721049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1332/2631] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:20:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:20:24.761534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:24.764642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:24.786954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0b2b4-3225-4978-8c82-29941f100772 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:24.797958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:24.815579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:24.815579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:24.815579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:24.815579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:24.817550 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:24.819316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:24.846899 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1300/2632] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:24.866057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:24.867944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:24.905134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0477f-fe32-4a03-93d6-9313aa8ab52b service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:24.925247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:24.925991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:24.926352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:24.926544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:24.932027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:24.952546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:24.952546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c16aee-67c4-4ac7-b46a-ef940fab604d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:24.982897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1333/2633] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:32.967212 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:32.979452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0759e3b-8328-4914-ba7d-22a9861dfa8f service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:32.988348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d91922-8058-4236-b570-267ce566a556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:32.990770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d91922-8058-4236-b570-267ce566a556 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:32.991145 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d91922-8058-4236-b570-267ce566a556 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:32.991525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d91922-8058-4236-b570-267ce566a556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:32.994260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-35d91922-8058-4236-b570-267ce566a556 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:33.014626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d91922-8058-4236-b570-267ce566a556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:33.016444 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d91922-8058-4236-b570-267ce566a556 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:33.034682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1301/2634] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:34.959657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:43.025306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=25 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:43.934682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a93e1c-d1df-468e-acf8-1f933d0ff941 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:43.935182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a93e1c-d1df-468e-acf8-1f933d0ff941 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:43.935331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a93e1c-d1df-468e-acf8-1f933d0ff941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:43.956017 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-23a93e1c-d1df-468e-acf8-1f933d0ff941 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:43.957224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-23a93e1c-d1df-468e-acf8-1f933d0ff941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:43.964449 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1334/2635] 10.222.0.46 () {58 vars in 967 bytes} [Wed Sep 7 19:20:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:43.977401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d91922-8058-4236-b570-267ce566a556 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:43.989265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d91922-8058-4236-b570-267ce566a556 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:43.994204 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:43.994840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:43.995120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:43.995391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:44.003563 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:44.005133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:44.014645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1302/2636] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:20:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:20:44.801865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:44.802168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:44.802425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:44.821310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:20:44.822568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:44.830380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1335/2637] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Sep 7 19:20:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:20:44.843641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:44.854160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f6d7-4fa3-44e3-9828-f255f24c5847 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:44.859246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:44.859834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:44.860121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:44.860383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:44.868555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:44.870134 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:44.878813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1303/2638] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:20:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:20:54.340897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:54.353548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e9834-f11c-434a-9ab7-a37455a91a1b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:54.358499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:54.360638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:54.360638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:54.360638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:54.361629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:54.374010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:54.375662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:54.386267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1336/2639] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:20:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:20:54.461348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:54.463802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:54.479079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab94d6-dab3-479d-adfa-af787369c9c1 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:54.483731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:54.484350 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:54.484667 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:54.484963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] PATH_INFO: `/v3/users/ddd54dd8bc3b41f3b649c2a26e5b54fe` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:54.486146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddd54dd8bc3b41f3b649c2a26e5b54fe)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:54.488072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:54.646097 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddd54dd8bc3b41f3b649c2a26e5b54fe'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:54.646770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddd54dd8bc3b41f3b649c2a26e5b54fe'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:54.647465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddd54dd8bc3b41f3b649c2a26e5b54fe'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:54.654952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddd54dd8bc3b41f3b649c2a26e5b54fe'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:54.666995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddd54dd8bc3b41f3b649c2a26e5b54fe'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:54.752471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1304/2640] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:20:54 2022] DELETE /identity/v3/users/ddd54dd8bc3b41f3b649c2a26e5b54fe => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:20:55.026940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:55.038033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc89a5-5701-4a90-9b40-a8b6d9f9419e service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:55.096995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:55.097622 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:55.098005 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:55.098296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] PATH_INFO: `/v3/projects/7e249e6872c846beaefe3f9b760b6319` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:55.099462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e249e6872c846beaefe3f9b760b6319)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:55.101347 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:55.203793 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e249e6872c846beaefe3f9b760b6319', 'request_id': 'req-c5ddbeab-d345-49f5-bf1d-060ae13143cf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.277600 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e249e6872c846beaefe3f9b760b6319', 'request_id': 'req-c5ddbeab-d345-49f5-bf1d-060ae13143cf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.284727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e249e6872c846beaefe3f9b760b6319', 'request_id': 'req-c5ddbeab-d345-49f5-bf1d-060ae13143cf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.295700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] The token cache is being invalidate because project 7e249e6872c846beaefe3f9b760b6319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:20:55.295981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.296592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.298037 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1337/2641] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:20:55 2022] DELETE /identity/v3/projects/7e249e6872c846beaefe3f9b760b6319 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:20:55.302270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:55.303728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:55.317336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07fc6d-883b-4f2d-91d9-6d35cc0f3dcc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:55.372774 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:55.373419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:55.373699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:55.373971 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] PATH_INFO: `/v3/users/251d37e79f5143deaa37f45a52012aa6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:55.375115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=251d37e79f5143deaa37f45a52012aa6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:55.377024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:55.553997 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '251d37e79f5143deaa37f45a52012aa6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.554660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '251d37e79f5143deaa37f45a52012aa6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.555177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '251d37e79f5143deaa37f45a52012aa6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.556371 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:55.562764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '251d37e79f5143deaa37f45a52012aa6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.575167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9569cc4e-c34e-496f-a725-c66fabe5daa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '251d37e79f5143deaa37f45a52012aa6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:55.648248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1305/2642] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:20:55 2022] DELETE /identity/v3/users/251d37e79f5143deaa37f45a52012aa6 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:20:55.873323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:20:55.874940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:20:55.889750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddbeab-d345-49f5-bf1d-060ae13143cf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:20:55.950380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:20:55.951634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:20:55.951913 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:20:55.952183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] PATH_INFO: `/v3/projects/fc9b2950d72c414fb68c055b15deba85` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:20:55.953375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9b2950d72c414fb68c055b15deba85)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:20:55.955304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:20:55.962163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:20:56.039235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9b2950d72c414fb68c055b15deba85', 'request_id': 'req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:56.088700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9b2950d72c414fb68c055b15deba85', 'request_id': 'req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:56.095700 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9b2950d72c414fb68c055b15deba85', 'request_id': 'req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:56.105863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] The token cache is being invalidate because project fc9b2950d72c414fb68c055b15deba85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:20:56.106147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:56.106795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:20:56.108906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1338/2643] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:20:55 2022] DELETE /identity/v3/projects/fc9b2950d72c414fb68c055b15deba85 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:21:00.287303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f447a-7ff1-4ba2-8781-0109e67702a8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:00.287797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f447a-7ff1-4ba2-8781-0109e67702a8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:00.287906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f447a-7ff1-4ba2-8781-0109e67702a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:00.307391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-376f447a-7ff1-4ba2-8781-0109e67702a8 None None] MFA Rules not processed for user `1da0e933ee954ba7be23a1997e4e5735`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:00.308707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-376f447a-7ff1-4ba2-8781-0109e67702a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:00.362688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1306/2644] 10.222.0.46 () {58 vars in 967 bytes} [Wed Sep 7 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:21:00.375408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:00.376783 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:00.444566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6eede-ccf9-4e92-ad06-df2ac918ac93 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:00.449608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42374efb278647738c5834bdb80db3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:00.450207 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:00.450486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:00.450789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:00.459268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:00.460839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:00.469560 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1339/2645] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Sep 7 19:21:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 19:21:08.062888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:08.064609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:08.064609 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:08.082481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] MFA Rules not processed for user `f9db5f73350c460f99e4ab5cdbe6907a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:08.083706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:08.139698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1307/2646] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:21:08.452964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:08.454374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:08.511900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f308-93d1-438d-8743-7f4beeeeed2b service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:08.515542 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:08.516151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:08.516430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:08.516704 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:08.518468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:08.528755 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:08.530337 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:08.538531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1340/2647] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:21:18.133059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=62,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:21:18.534838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:21:45.064611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:45.066136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:45.078963 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e81e3-4285-4ad2-8393-d49646b61610 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:45.082642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:45.083237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:45.083518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:45.083788 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:45.085463 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:45.150786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:45.150786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:45.159924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1308/2648] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:21:48.689076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:48.693815 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:48.705325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59664ca7-ab0c-4d59-a8eb-7d7fa979b207 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:48.710125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:48.710775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:48.711073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:48.711343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] PATH_INFO: `/v3/users/920bdd998f0c4b62b108afc799a43964` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:48.712466 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=920bdd998f0c4b62b108afc799a43964)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:48.714392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:48.903401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920bdd998f0c4b62b108afc799a43964'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:48.904266 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920bdd998f0c4b62b108afc799a43964'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:48.904941 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920bdd998f0c4b62b108afc799a43964'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:48.914424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920bdd998f0c4b62b108afc799a43964'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:48.930423 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31b55656-efb9-4329-9ebc-a20ba48da05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920bdd998f0c4b62b108afc799a43964'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:48.992233 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1341/2649] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:21:48 2022] DELETE /identity/v3/users/920bdd998f0c4b62b108afc799a43964 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:21:49.227262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:49.239682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54019243-ba7a-405b-918c-e68ca674f9e3 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:49.300888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:49.301517 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.301797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.302072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] PATH_INFO: `/v3/projects/8a7d9bc5395b4fa09a1024d20a980991` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.303206 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7d9bc5395b4fa09a1024d20a980991)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:49.305118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:49.402889 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7d9bc5395b4fa09a1024d20a980991', 'request_id': 'req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:49.409446 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7d9bc5395b4fa09a1024d20a980991', 'request_id': 'req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:49.414946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7d9bc5395b4fa09a1024d20a980991', 'request_id': 'req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:49.482264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] The token cache is being invalidate because project 8a7d9bc5395b4fa09a1024d20a980991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:21:49.482488 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:49.483138 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8b4d27b6-c0b7-41a2-bce0-fcfce2863b8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:21:49.485246 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1309/2650] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:21:49 2022] DELETE /identity/v3/projects/8a7d9bc5395b4fa09a1024d20a980991 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:21:49.490335 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.490680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.490956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.510147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:49.511369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:49.568792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1342/2651] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:21:49.576876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.577161 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.577440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.597443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:49.598688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:49.606533 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1310/2652] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:21:49.611540 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:49.621630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09cbf7-8acf-4b6d-a6d7-34e064d154c5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:49.626272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:49.626899 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.627180 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.627450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.628095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:21:49.628400 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:49.629990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:49.641936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1343/2653] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:21:49.646125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:49.655575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d978bd-82d7-456e-80de-b9e97f471f95 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:49.660805 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:49.661397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.661678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.661952 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.662614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:49.664516 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:49.679989 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:21:49.775614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1311/2654] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:21:49 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:21:49.780360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:49.790916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b9d34-7146-4a69-96d4-e147e6f94e05 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:49.848238 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:49.848832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.849108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.849381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.850034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:49.851957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:49.859949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:21:49.962880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1344/2655] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:21:49 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:21:49.967122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:49.978292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ced09-d077-45e0-abfe-1762d0f297c5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:49.983182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:49.983803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:49.984080 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:49.984404 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:49.985033 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:49.986720 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:49.996596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1312/2656] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:21:50.000895 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:50.010532 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c09ae-21e9-4f4b-9d02-7bd8b7ad44fd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:50.015139 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28e6cc-e14f-4ce4-be97-56f9fc016183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:50.018757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28e6cc-e14f-4ce4-be97-56f9fc016183 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:50.018757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28e6cc-e14f-4ce4-be97-56f9fc016183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:50.018757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28e6cc-e14f-4ce4-be97-56f9fc016183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:50.018757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28e6cc-e14f-4ce4-be97-56f9fc016183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:50.019163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28e6cc-e14f-4ce4-be97-56f9fc016183 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:50.029681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1345/2657] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:21:50.033687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:50.044128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f36ed-4ff2-4801-81ad-244ec9b3e464 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:50.048815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:50.049464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:50.049744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:50.050018 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] PATH_INFO: `/v3/projects/56c66a304fdf4e9eba6ca0b7a15f492b/users/c639d3e06c4c4fcc834979684b00af6c/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:50.064906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c66a304fdf4e9eba6ca0b7a15f492b, user_id=c639d3e06c4c4fcc834979684b00af6c, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:50.067636 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:50.146456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1313/2658] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:21:50 2022] PUT /identity/v3/projects/56c66a304fdf4e9eba6ca0b7a15f492b/users/c639d3e06c4c4fcc834979684b00af6c/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:21:50.151678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953fc86-6848-4c4e-bdfd-f8cf90be1a37 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:50.151960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953fc86-6848-4c4e-bdfd-f8cf90be1a37 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:50.152257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953fc86-6848-4c4e-bdfd-f8cf90be1a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:50.171237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c953fc86-6848-4c4e-bdfd-f8cf90be1a37 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:50.173662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c953fc86-6848-4c4e-bdfd-f8cf90be1a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:50.230772 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1346/2659] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:21:50.240474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:50.242153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:50.299642 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7ec51-e3a3-4203-acf9-44e80ebe9b45 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:50.303232 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:50.303830 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:50.304105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:50.304376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:50.313771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:50.315222 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:50.324707 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1314/2660] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:21:55.487498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5d2a7-b9da-4f85-b4c4-8a9c4eeee556 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:55.488007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5d2a7-b9da-4f85-b4c4-8a9c4eeee556 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:55.488106 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5d2a7-b9da-4f85-b4c4-8a9c4eeee556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:55.517319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-05e5d2a7-b9da-4f85-b4c4-8a9c4eeee556 None None] MFA Rules not processed for user `c639d3e06c4c4fcc834979684b00af6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:55.518589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05e5d2a7-b9da-4f85-b4c4-8a9c4eeee556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:55.592838 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1347/2661] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:21:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:21:55.610297 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:21:55.611792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:55.689329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ec7d9-594f-474c-9104-ff4e64f7fecb service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:21:55.696535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923f689-2e02-46e5-a580-415ac7c18e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:21:55.697160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923f689-2e02-46e5-a580-415ac7c18e70 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:55.697441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923f689-2e02-46e5-a580-415ac7c18e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:55.697717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923f689-2e02-46e5-a580-415ac7c18e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:55.707063 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923f689-2e02-46e5-a580-415ac7c18e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:21:55.708797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923f689-2e02-46e5-a580-415ac7c18e70 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:21:55.720341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1315/2662] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:21:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:21:57.975598 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c74549-24ea-4e64-808d-aa75dec5d4b0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:21:57.976084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c74549-24ea-4e64-808d-aa75dec5d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:21:57.976181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c74549-24ea-4e64-808d-aa75dec5d4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:21:57.995456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-99c74549-24ea-4e64-808d-aa75dec5d4b0 None None] MFA Rules not processed for user `c639d3e06c4c4fcc834979684b00af6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:21:57.996665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-99c74549-24ea-4e64-808d-aa75dec5d4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:21:58.004338 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1348/2663] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:22:05.714181 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:22:07.990002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:22:38.945818 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d1110-3fe0-4da2-ba2b-dd899d72c8a3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:22:38.946286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d1110-3fe0-4da2-ba2b-dd899d72c8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:22:38.946395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d1110-3fe0-4da2-ba2b-dd899d72c8a3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:22:38.947853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1316/2664] 10.222.0.46 () {54 vars in 961 bytes} [Wed Sep 7 19:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:22:38.952858 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:22:38.953301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:22:38.953416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:22:39.034740 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] MFA Rules not processed for user `c32b1292d0c6436ba4669dbfd1c73ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:22:39.035938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:22:39.141819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1349/2665] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Sep 7 19:22:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:22:39.149942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d1110-3fe0-4da2-ba2b-dd899d72c8a3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:22:39.163474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d1110-3fe0-4da2-ba2b-dd899d72c8a3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:22:39.168391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:22:39.168920 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:22:39.169192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:22:39.169461 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:22:39.171172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:22:39.182346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:22:39.183933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cc53f-b108-403d-a20c-dc504877f861 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:22:39.194078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1317/2666] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:22:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:22:49.140132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=74 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:22:49.189995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:22:59.935112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:22:59.936545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:23:00.019852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf3e5a-5084-40f7-be3b-dd3cdf8c38b3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:23:00.023765 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:23:00.024476 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:23:00.024759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:23:00.025042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:23:00.027083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:23:00.091652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:23:00.093228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0436f5-b1b8-46c5-ab99-5f4e4c4d6944 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:23:00.102779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1350/2667] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:22:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:23:00.822201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7a03a-e8f1-4832-a130-f0a76cd18969 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:23:00.822201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7a03a-e8f1-4832-a130-f0a76cd18969 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:23:00.822201 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7a03a-e8f1-4832-a130-f0a76cd18969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:23:00.843375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c0d7a03a-e8f1-4832-a130-f0a76cd18969 None None] MFA Rules not processed for user `c639d3e06c4c4fcc834979684b00af6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:23:00.844584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c0d7a03a-e8f1-4832-a130-f0a76cd18969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:23:00.853379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1318/2668] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:23:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:23:08.679694 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b545f7-8131-424e-b54a-2acb222cf69f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:23:08.680293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b545f7-8131-424e-b54a-2acb222cf69f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:23:08.680293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b545f7-8131-424e-b54a-2acb222cf69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:23:08.701997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c1b545f7-8131-424e-b54a-2acb222cf69f None None] MFA Rules not processed for user `f9db5f73350c460f99e4ab5cdbe6907a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:23:08.703267 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1b545f7-8131-424e-b54a-2acb222cf69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:23:08.716394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1351/2669] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:23:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:23:10.837475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:23:13.975780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:23:13.976423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:23:13.976423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:23:13.996061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] MFA Rules not processed for user `7c8142614fe149fd9593cfba97ecad18`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:23:13.997469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:23:14.059910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1319/2670] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:23:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:23:18.696016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=60 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:23:24.047944 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=25 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:19.705995 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:19.706516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:19.706659 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:19.726451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] MFA Rules not processed for user `c639d3e06c4c4fcc834979684b00af6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:19.732498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:19.735126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1352/2671] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:29.720738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:48.648324 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:48.649806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:48.698213 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae552a-13dc-4f94-b921-c34b4b5457cc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:48.703132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:48.708670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:48.708670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:48.708670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] PATH_INFO: `/v3/users/7c8142614fe149fd9593cfba97ecad18` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:48.708670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c8142614fe149fd9593cfba97ecad18)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:48.708670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:48.933306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c8142614fe149fd9593cfba97ecad18'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:48.933880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c8142614fe149fd9593cfba97ecad18'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:48.934500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c8142614fe149fd9593cfba97ecad18'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:48.941442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c8142614fe149fd9593cfba97ecad18'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:48.953846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7e364d5e-c36e-4a3d-830a-74046f7d09a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c8142614fe149fd9593cfba97ecad18'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:49.029595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1320/2672] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:24:48 2022] DELETE /identity/v3/users/7c8142614fe149fd9593cfba97ecad18 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:49.264934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:49.278059 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea11eb1-21b2-407f-a31f-353ae9dd3032 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:49.334369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:49.335029 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:49.335308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:49.335582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] PATH_INFO: `/v3/projects/53d22963975b4bb891b1858f4afe3275` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:49.336703 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d22963975b4bb891b1858f4afe3275)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:49.338628 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:49.441113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d22963975b4bb891b1858f4afe3275', 'request_id': 'req-97339108-0443-4954-a81d-7bf61737646c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:49.508374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d22963975b4bb891b1858f4afe3275', 'request_id': 'req-97339108-0443-4954-a81d-7bf61737646c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:49.514998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d22963975b4bb891b1858f4afe3275', 'request_id': 'req-97339108-0443-4954-a81d-7bf61737646c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:49.525599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] The token cache is being invalidate because project 53d22963975b4bb891b1858f4afe3275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:24:49.525885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:49.526510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-97339108-0443-4954-a81d-7bf61737646c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:49.528253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1353/2673] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:24:49 2022] DELETE /identity/v3/projects/53d22963975b4bb891b1858f4afe3275 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:49.533180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:49.533468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:49.533770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:49.583646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:49.584872 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:49.643334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1321/2674] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:49.651605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:49.651889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:49.652418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:49.672504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:49.673799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:49.682355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1354/2675] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:49.686692 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:49.697579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce0d48-9130-4b5a-bfcd-bb39b40d3dff None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:49.702305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:49.702924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:49.703205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:49.703480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:49.704128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:24:49.704427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:49.706024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:49.717683 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1322/2676] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:24:49.721989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:49.731579 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ccbd1-c20a-4b75-a2a5-459897c7643a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:49.736156 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:49.736737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:49.737012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:49.737277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:49.737901 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:49.739807 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:49.869665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1355/2677] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:24:49 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:49.874011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:49.884120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535732ab-0300-4d1c-8b84-faaa98dc2bd2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:49.888762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:49.940945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:49.941582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:49.941860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:49.942136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:49.942811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:49.944721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.051418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1323/2678] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:24:49 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:24:50.055711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.066028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e023-d4c4-4000-8422-0659f63dc213 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.070611 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.071203 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.071480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.071752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.072388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.074011 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.084274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1356/2679] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:50.088270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.098313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2f19-7738-421e-b431-4e94f7c1c61d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.103574 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9808e75-bad9-4941-9a42-6cc7e39262d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.104166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9808e75-bad9-4941-9a42-6cc7e39262d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.104442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9808e75-bad9-4941-9a42-6cc7e39262d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.104714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9808e75-bad9-4941-9a42-6cc7e39262d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.105355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9808e75-bad9-4941-9a42-6cc7e39262d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.107112 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9808e75-bad9-4941-9a42-6cc7e39262d8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.117546 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1324/2680] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:50.121678 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.132012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d7c58-8cbe-47ab-a5c9-9a4e940809d2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.136590 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.137217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.137492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.137764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] PATH_INFO: `/v3/projects/16324b3e89284c03be4323127489df53/users/157bd7e2c2394c7687762e53e7851f60/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.153253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16324b3e89284c03be4323127489df53, user_id=157bd7e2c2394c7687762e53e7851f60, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.155925 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.251644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1357/2681] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:24:50 2022] PUT /identity/v3/projects/16324b3e89284c03be4323127489df53/users/157bd7e2c2394c7687762e53e7851f60/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:50.262339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.262654 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.262925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.291113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] MFA Rules not processed for user `157bd7e2c2394c7687762e53e7851f60`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:50.292358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:50.355238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1325/2682] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:50.359855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.378792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:50.422279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd0dc-8cb2-4091-8441-6f1d88c3355f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.427523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.428125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.428403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.428676 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.429318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.431380 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.498247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1358/2683] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:24:50 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:50.502182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.513329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df628681-2abd-464f-9f46-f5945d6b2a2b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.569547 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.570147 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.570430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.570728 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.571364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.573268 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.657438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1326/2684] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:24:50 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:50.661369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.672201 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a9046-b186-450f-a5a4-0d577aa64c64 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.676845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.677553 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.677831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.678103 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.678868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.680377 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.690829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1359/2685] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:50.694966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.705007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb416e-2fcf-4f30-8fcb-b9fb7f7cc51d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.710370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.711047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.711320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.711586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] PATH_INFO: `/v3/projects/84efae8c840e415998703374292e15a4/users/9760c4f9e93941d9af2422d19591296a/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.713734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:50.727482 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84efae8c840e415998703374292e15a4, user_id=9760c4f9e93941d9af2422d19591296a, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.730205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.794381 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1327/2686] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:24:50 2022] PUT /identity/v3/projects/84efae8c840e415998703374292e15a4/users/9760c4f9e93941d9af2422d19591296a/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:50.798375 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.808456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dd9b8-2472-4a7d-aca3-e376a98eff1d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.863349 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda46dec-28f6-4aaa-a902-69679520476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.863943 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda46dec-28f6-4aaa-a902-69679520476c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.864220 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda46dec-28f6-4aaa-a902-69679520476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.864494 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda46dec-28f6-4aaa-a902-69679520476c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.865149 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda46dec-28f6-4aaa-a902-69679520476c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.866739 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda46dec-28f6-4aaa-a902-69679520476c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.876318 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1360/2687] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:50.880265 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:50.891143 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9398f92-e554-43e7-b6df-6a539bac6c99 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:50.895738 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:50.896912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.897186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.897452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] PATH_INFO: `/v3/projects/84efae8c840e415998703374292e15a4/users/9760c4f9e93941d9af2422d19591296a/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.899386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84efae8c840e415998703374292e15a4, user_id=9760c4f9e93941d9af2422d19591296a, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:50.901996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:50.912299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1328/2688] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:24:50 2022] PUT /identity/v3/projects/84efae8c840e415998703374292e15a4/users/9760c4f9e93941d9af2422d19591296a/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:50.923526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:50.923810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:50.924082 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:50.952451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] MFA Rules not processed for user `9760c4f9e93941d9af2422d19591296a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:50.953648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:51.019623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1361/2689] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:51.042128 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:51.043862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:51.100353 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f0d4-a76d-4a23-8b4c-7a291a169cd1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:51.103981 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aef065-7539-4185-89be-95fbe95a9ce7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:51.104575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aef065-7539-4185-89be-95fbe95a9ce7 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:51.104850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aef065-7539-4185-89be-95fbe95a9ce7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:51.105120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aef065-7539-4185-89be-95fbe95a9ce7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:51.113927 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aef065-7539-4185-89be-95fbe95a9ce7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:51.115378 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aef065-7539-4185-89be-95fbe95a9ce7 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:51.124701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1329/2690] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:24:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:24:51.879306 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:51.893187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:51.942986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c32d2-fb24-4c87-b555-dab1fff9a87b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:51.947523 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:51.948143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:51.948422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:51.948695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] PATH_INFO: `/v3/users/157bd7e2c2394c7687762e53e7851f60` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:51.949814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=157bd7e2c2394c7687762e53e7851f60)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:51.951744 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:52.190839 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '157bd7e2c2394c7687762e53e7851f60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:52.191500 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '157bd7e2c2394c7687762e53e7851f60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:52.192113 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '157bd7e2c2394c7687762e53e7851f60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:52.200792 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '157bd7e2c2394c7687762e53e7851f60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:52.216458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '157bd7e2c2394c7687762e53e7851f60'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:52.296920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1362/2691] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:24:51 2022] DELETE /identity/v3/users/157bd7e2c2394c7687762e53e7851f60 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:52.301457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:52.301928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:52.314475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:52.321942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:52.323226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:52.381710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1330/2692] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:52.391629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:52.393016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:52.472447 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fd7df-bdbc-4f9b-884f-c5687a1a6382 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:52.475906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:52.476554 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:52.476825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:52.477114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:52.485965 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:52.487393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:52.496638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1363/2693] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:24:52.909946 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:52.911423 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:52.923588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002299a-9215-4bee-93c0-7589ff2b2ddc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:52.928663 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:52.929292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:52.929568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:52.929866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] PATH_INFO: `/v3/projects/16324b3e89284c03be4323127489df53` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:52.930987 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16324b3e89284c03be4323127489df53)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:52.932896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:53.054275 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16324b3e89284c03be4323127489df53', 'request_id': 'req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.060635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16324b3e89284c03be4323127489df53', 'request_id': 'req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.066230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16324b3e89284c03be4323127489df53', 'request_id': 'req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.069851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:53.142347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] The token cache is being invalidate because project 16324b3e89284c03be4323127489df53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:24:53.142708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.143375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.145991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1331/2694] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:24:52 2022] DELETE /identity/v3/projects/16324b3e89284c03be4323127489df53 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:53.150537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:53.160510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f75f0-344c-4dd5-8890-e4aa9bb027a3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:53.171951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:53.217916 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:53.218576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:53.218854 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:53.219131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] PATH_INFO: `/v3/users/9760c4f9e93941d9af2422d19591296a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:53.220324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9760c4f9e93941d9af2422d19591296a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:53.222245 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:53.379032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9760c4f9e93941d9af2422d19591296a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.379682 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9760c4f9e93941d9af2422d19591296a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.380292 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9760c4f9e93941d9af2422d19591296a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.387823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9760c4f9e93941d9af2422d19591296a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.400226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5534c19f-f90c-47a5-a744-05a1476a3884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9760c4f9e93941d9af2422d19591296a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:53.451806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1364/2695] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:24:53 2022] DELETE /identity/v3/users/9760c4f9e93941d9af2422d19591296a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:53.864096 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:53.866074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:53.881660 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3ccbe-932f-4c28-a4de-f0cd781fbb67 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:53.939710 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:53.940341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:53.940617 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:53.940913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] PATH_INFO: `/v3/projects/84efae8c840e415998703374292e15a4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:53.942023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84efae8c840e415998703374292e15a4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:53.943999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.079500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84efae8c840e415998703374292e15a4', 'request_id': 'req-4b49884d-7264-420d-9d2c-9bd071581706'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:54.085888 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84efae8c840e415998703374292e15a4', 'request_id': 'req-4b49884d-7264-420d-9d2c-9bd071581706'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:54.091841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84efae8c840e415998703374292e15a4', 'request_id': 'req-4b49884d-7264-420d-9d2c-9bd071581706'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:54.158898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] The token cache is being invalidate because project 84efae8c840e415998703374292e15a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:24:54.159116 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:54.159722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4b49884d-7264-420d-9d2c-9bd071581706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:54.161239 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1332/2696] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:24:53 2022] DELETE /identity/v3/projects/84efae8c840e415998703374292e15a4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:54.166665 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.172644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.172644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.187390 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:54.188599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:54.252094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1365/2697] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:54.260025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.260303 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.260605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.281177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:54.282364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:54.290877 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1333/2698] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:54.295135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.304921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070977-e1c8-45a2-8399-a5a9d36314fe None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.309831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.310455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.310836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.311115 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.311767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:24:54.312073 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.313743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.325371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1366/2699] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:24:54.329760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.339490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60017257-2b19-46a8-9c6a-657d49770250 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.344034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.344619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.344893 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.345159 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.345784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.347699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.423595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1334/2700] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:54.428098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.438044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb163aa-b8ea-4690-a6ca-eee3d255d8e0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.456482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:54.494126 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.494767 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.495042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.495359 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.496004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.498818 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.582394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1367/2701] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:24:54 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:54.587070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.597475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765bd60-4d99-4bdc-85c7-8508d44b2417 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.602969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.603602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.603929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.604195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.604840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.606431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.616532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1335/2702] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:54.620389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.631077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ff4e9-428b-4bdd-87c0-f09e6d6ba3e1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.635734 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a3cf0-331f-4b8f-8504-e7eebe6387cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.636325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a3cf0-331f-4b8f-8504-e7eebe6387cd admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.636602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a3cf0-331f-4b8f-8504-e7eebe6387cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.636874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a3cf0-331f-4b8f-8504-e7eebe6387cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.637512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a3cf0-331f-4b8f-8504-e7eebe6387cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.639117 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a3cf0-331f-4b8f-8504-e7eebe6387cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.650250 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1368/2703] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:54.654786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.664706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254785-ce61-490c-9c46-195afd9bd22c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.669490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.670140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.670420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.670729 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] PATH_INFO: `/v3/projects/4d0f431dd4be42aa93fb4a3a99c5ad41/users/508ada518bef4bd8b6bc74c0065e9a5c/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.685182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f431dd4be42aa93fb4a3a99c5ad41, user_id=508ada518bef4bd8b6bc74c0065e9a5c, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.687813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:54.760314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1336/2704] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:24:54 2022] PUT /identity/v3/projects/4d0f431dd4be42aa93fb4a3a99c5ad41/users/508ada518bef4bd8b6bc74c0065e9a5c/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:54.770751 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.771645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.771941 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.799108 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] MFA Rules not processed for user `508ada518bef4bd8b6bc74c0065e9a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:54.800308 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:54.862844 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1369/2705] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:24:54.867327 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:54.892787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:54.929028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c411e8f-9a8b-4146-9761-9e5855766f38 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:54.933718 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:54.934309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:54.934611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:54.934884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:54.935585 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:54.937424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.018048 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1337/2706] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:55.022785 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.033483 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53f9f3-b267-4da5-9dd9-03d6db2a5b78 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.089846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.090453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.090760 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.091031 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.091677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.093575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.200155 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1370/2707] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:24:55 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:24:55.204104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.215171 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651f107-718e-42bf-af40-51dba45a1635 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.219817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.220440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.220693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.220964 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.221662 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.223291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.234397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1338/2708] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:55.238724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.249114 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb875dd-4c30-4552-8868-79e401b7a492 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.253746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.254392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.254712 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.254992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] PATH_INFO: `/v3/projects/14500072e39743e5b3c3bfc7678f5a0a/users/cc26fc8e83e54eb7ad6262287247be68/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.269681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14500072e39743e5b3c3bfc7678f5a0a, user_id=cc26fc8e83e54eb7ad6262287247be68, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.272840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.275743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:55.359587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1371/2709] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:24:55 2022] PUT /identity/v3/projects/14500072e39743e5b3c3bfc7678f5a0a/users/cc26fc8e83e54eb7ad6262287247be68/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:24:55.363701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.373762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a15eb-5e7f-4360-bb74-9d06763712e5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.428594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75990230-52f9-4d41-8d98-cd46e61ae9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.429188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990230-52f9-4d41-8d98-cd46e61ae9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.429463 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990230-52f9-4d41-8d98-cd46e61ae9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.429735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990230-52f9-4d41-8d98-cd46e61ae9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.430365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75990230-52f9-4d41-8d98-cd46e61ae9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.431999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75990230-52f9-4d41-8d98-cd46e61ae9ce admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.441323 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1339/2710] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:24:55.446083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.456320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57cd2a-4ca5-4160-85f1-23cdbdf88cf5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.461395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.462023 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.462296 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.462654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] PATH_INFO: `/v3/projects/14500072e39743e5b3c3bfc7678f5a0a/users/cc26fc8e83e54eb7ad6262287247be68/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.464549 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14500072e39743e5b3c3bfc7678f5a0a, user_id=cc26fc8e83e54eb7ad6262287247be68, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.467298 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.479294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1372/2711] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:24:55 2022] PUT /identity/v3/projects/14500072e39743e5b3c3bfc7678f5a0a/users/cc26fc8e83e54eb7ad6262287247be68/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:24:55.490458 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.490775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.491057 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.518590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] MFA Rules not processed for user `cc26fc8e83e54eb7ad6262287247be68`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:55.519803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:55.591151 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1340/2712] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:55.602367 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.604120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:55.661864 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f628d60-9b71-4168-a772-5a2910b0e25b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.665565 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.666163 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.666455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.669182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.675996 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.677629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.687394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1373/2713] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:24:55.899755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:55.910530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0ab38-f428-4e06-bdc4-232c8e188536 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:55.914852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284c053-7190-4751-939f-a8a5ab0a66a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:55.920491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284c053-7190-4751-939f-a8a5ab0a66a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:55.920491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284c053-7190-4751-939f-a8a5ab0a66a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:55.920491 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284c053-7190-4751-939f-a8a5ab0a66a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:55.929397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:55.973123 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284c053-7190-4751-939f-a8a5ab0a66a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:55.974576 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284c053-7190-4751-939f-a8a5ab0a66a3 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:55.983874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1341/2714] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:24:59.418595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:59.480735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6e601-1d4a-4568-a534-0a19975c651a service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:59.485179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:59.485791 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:59.486091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:59.486358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] PATH_INFO: `/v3/users/508ada518bef4bd8b6bc74c0065e9a5c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:59.487811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508ada518bef4bd8b6bc74c0065e9a5c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:59.489530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:59.687368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508ada518bef4bd8b6bc74c0065e9a5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:59.688012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508ada518bef4bd8b6bc74c0065e9a5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:59.688612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508ada518bef4bd8b6bc74c0065e9a5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:59.695605 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508ada518bef4bd8b6bc74c0065e9a5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:59.708282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508ada518bef4bd8b6bc74c0065e9a5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:24:59.777860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1374/2715] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:24:59 2022] DELETE /identity/v3/users/508ada518bef4bd8b6bc74c0065e9a5c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:24:59.782376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:59.782860 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:59.782990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:59.802506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:24:59.803796 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:59.861735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1342/2716] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:24:59.871469 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:24:59.872856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:24:59.879393 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:24:59.931942 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8132-4f31-49cd-a21f-0a61b1e746a0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:24:59.936239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:24:59.937121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:24:59.937492 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:24:59.937778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:24:59.948060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:24:59.949482 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:24:59.961497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1375/2717] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:00.619034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:00.620492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:00.638735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65afac2-04cb-4828-887a-1671fe3a0f24 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:00.640215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:00.640887 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:00.641296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:00.641575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] PATH_INFO: `/v3/projects/4d0f431dd4be42aa93fb4a3a99c5ad41` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:00.642749 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0f431dd4be42aa93fb4a3a99c5ad41)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:00.644918 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:00.773122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0f431dd4be42aa93fb4a3a99c5ad41', 'request_id': 'req-dfa91159-4cdb-4beb-93a7-55df31a09b85'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:00.779453 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0f431dd4be42aa93fb4a3a99c5ad41', 'request_id': 'req-dfa91159-4cdb-4beb-93a7-55df31a09b85'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:00.785066 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0f431dd4be42aa93fb4a3a99c5ad41', 'request_id': 'req-dfa91159-4cdb-4beb-93a7-55df31a09b85'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:00.878735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] The token cache is being invalidate because project 4d0f431dd4be42aa93fb4a3a99c5ad41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:00.879028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:00.879657 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:00.881668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1343/2718] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:00 2022] DELETE /identity/v3/projects/4d0f431dd4be42aa93fb4a3a99c5ad41 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:00.891924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:00.902424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e77ea-daf5-40f1-9eb3-31cac519b39d service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:00.958184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:00.958826 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:00.959105 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:00.959378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] PATH_INFO: `/v3/users/cc26fc8e83e54eb7ad6262287247be68` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:00.960480 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc26fc8e83e54eb7ad6262287247be68)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:00.962371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:01.144433 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc26fc8e83e54eb7ad6262287247be68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.145371 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc26fc8e83e54eb7ad6262287247be68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.146603 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc26fc8e83e54eb7ad6262287247be68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.153952 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc26fc8e83e54eb7ad6262287247be68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.166616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7acf5da-b400-4edc-b9ec-08a33ab93fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc26fc8e83e54eb7ad6262287247be68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.239731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1376/2719] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:00 2022] DELETE /identity/v3/users/cc26fc8e83e54eb7ad6262287247be68 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:01.595691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:01.597184 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:01.610439 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91159-4cdb-4beb-93a7-55df31a09b85 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:01.649696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:01.668254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:01.668892 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:01.669174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:01.669451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] PATH_INFO: `/v3/projects/14500072e39743e5b3c3bfc7678f5a0a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:01.670577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] RBAC: Authorizing `identity:delete_project(project_id=14500072e39743e5b3c3bfc7678f5a0a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:01.672790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:01.800611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14500072e39743e5b3c3bfc7678f5a0a', 'request_id': 'req-f0e10be4-13c5-43f1-a937-d00e1e73187f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.807162 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14500072e39743e5b3c3bfc7678f5a0a', 'request_id': 'req-f0e10be4-13c5-43f1-a937-d00e1e73187f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.813335 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14500072e39743e5b3c3bfc7678f5a0a', 'request_id': 'req-f0e10be4-13c5-43f1-a937-d00e1e73187f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.918611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] The token cache is being invalidate because project 14500072e39743e5b3c3bfc7678f5a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:01.918939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.919579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f0e10be4-13c5-43f1-a937-d00e1e73187f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:01.921083 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1344/2720] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:01 2022] DELETE /identity/v3/projects/14500072e39743e5b3c3bfc7678f5a0a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:01.925980 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:01.926499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:01.926746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:01.946920 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:01.948040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:01.967776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:02.006121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1377/2721] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:02.017907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.018210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.018459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.039005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:02.040191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:02.048131 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1345/2722] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:02.053033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.062716 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f5410-2f23-462f-9157-64e84e182cc7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.067379 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.068623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.068905 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.069183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.069829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:25:02.070133 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.071737 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.083424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1378/2723] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:25:02.087678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.096883 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce512-1b80-49e7-836c-915616b935d9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.101575 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.102169 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.102448 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.102763 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.103400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.105304 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.195031 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1346/2724] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:02 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:02.199343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.213553 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc34fc6-3097-4bc6-b7de-9c1bf4e807b6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.268169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.268774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.269053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.269323 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.269949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.271876 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.383324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1379/2725] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:02 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:25:02.387731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.398823 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1ecd-1e48-4770-b4d5-b1d652345cbf admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.403732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.404329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.404607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.404882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.405558 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.407210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.417086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1347/2726] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:02.421125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.431068 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c1755-dcfc-4f6b-aa73-21ed63491242 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.435617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a2ce-6c4f-467e-922d-abadd0cfd67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.436202 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a2ce-6c4f-467e-922d-abadd0cfd67e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.436479 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a2ce-6c4f-467e-922d-abadd0cfd67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.436752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a2ce-6c4f-467e-922d-abadd0cfd67e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.437385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074a2ce-6c4f-467e-922d-abadd0cfd67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.439032 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074a2ce-6c4f-467e-922d-abadd0cfd67e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.449888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1380/2727] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:25:02.453815 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.464349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264511f-0ede-4ceb-971d-9036e7a69685 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.469035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.469690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.469990 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.470260 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] PATH_INFO: `/v3/projects/495ddf7c6d30465b8684811f793b001f/users/27bbe93cd56740ee847c815ad25d9d4d/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.484753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495ddf7c6d30465b8684811f793b001f, user_id=27bbe93cd56740ee847c815ad25d9d4d, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.487475 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.554183 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1348/2728] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:02 2022] PUT /identity/v3/projects/495ddf7c6d30465b8684811f793b001f/users/27bbe93cd56740ee847c815ad25d9d4d/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:02.617501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.617770 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.618039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.646151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] MFA Rules not processed for user `27bbe93cd56740ee847c815ad25d9d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:02.647343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:02.710499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1381/2729] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:02.715049 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.776319 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be73bf-392c-4d33-8721-9d4039114895 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.781245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.781837 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.782117 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.782390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.783053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.784942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:02.795726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:02.892645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1349/2730] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:02 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:25:02.896319 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:02.907165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d6e4-324a-4b63-ac9b-b96f95f4f70c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:02.925132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:02.962225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:02.962877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:02.963165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:02.963441 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:02.964112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:02.966054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.063351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1382/2731] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:02 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:03.067464 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:03.077886 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0d92-a1ad-4464-987e-68dd3552eab3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:03.082602 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:03.083193 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.083469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.083741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.084480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:03.086077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.095993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1350/2732] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:03.100531 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:03.110179 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7719b3-9598-4c36-8543-b2ff51be2ceb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:03.115363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:03.115994 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.116268 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.116541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] PATH_INFO: `/v3/projects/07c99aa3dc9f46c2ac8e52d143cd11b9/users/63d1b2a8233e4a289cf5e1fdaf865ef9/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.131048 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c99aa3dc9f46c2ac8e52d143cd11b9, user_id=63d1b2a8233e4a289cf5e1fdaf865ef9, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:03.133658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.228821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1383/2733] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:03 2022] PUT /identity/v3/projects/07c99aa3dc9f46c2ac8e52d143cd11b9/users/63d1b2a8233e4a289cf5e1fdaf865ef9/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:03.232916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:03.244594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2316441-d8bb-44e5-a77b-bbc87aa6580c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:03.301346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ce0b9-5c81-4b60-a5aa-36ec20f58047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:03.301967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ce0b9-5c81-4b60-a5aa-36ec20f58047 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.302247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ce0b9-5c81-4b60-a5aa-36ec20f58047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.302520 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ce0b9-5c81-4b60-a5aa-36ec20f58047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.303220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ce0b9-5c81-4b60-a5aa-36ec20f58047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:03.304811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ce0b9-5c81-4b60-a5aa-36ec20f58047 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.314278 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1351/2734] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:03.318572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:03.328953 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faf320-2e1c-4bc2-a6bb-7a5c0ea7fa96 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:03.334100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:03.334823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.335120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.335383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] PATH_INFO: `/v3/projects/07c99aa3dc9f46c2ac8e52d143cd11b9/users/63d1b2a8233e4a289cf5e1fdaf865ef9/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.337340 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c99aa3dc9f46c2ac8e52d143cd11b9, user_id=63d1b2a8233e4a289cf5e1fdaf865ef9, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:03.340000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.351424 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1384/2735] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:03 2022] PUT /identity/v3/projects/07c99aa3dc9f46c2ac8e52d143cd11b9/users/63d1b2a8233e4a289cf5e1fdaf865ef9/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:03.361443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.361731 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.362000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.388839 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] MFA Rules not processed for user `63d1b2a8233e4a289cf5e1fdaf865ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:03.390022 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:03.457544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1352/2736] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:03.469215 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:03.470896 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:03.527445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27554d70-dde3-4271-aeb4-142ba1194d3a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:03.530956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee4c79-9721-43f6-8919-686f7094d905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:03.531545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee4c79-9721-43f6-8919-686f7094d905 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.531816 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee4c79-9721-43f6-8919-686f7094d905 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.532084 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee4c79-9721-43f6-8919-686f7094d905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.541425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee4c79-9721-43f6-8919-686f7094d905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:03.542836 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee4c79-9721-43f6-8919-686f7094d905 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.552052 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1385/2737] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:03.580434 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:03.645132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de89db7-4b53-4a2a-9f6b-efaa94e72504 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:03.649785 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:03.650410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.650722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.651005 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] PATH_INFO: `/v3/users/27bbe93cd56740ee847c815ad25d9d4d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.652388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27bbe93cd56740ee847c815ad25d9d4d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:03.654312 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:03.656891 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:03.887776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27bbe93cd56740ee847c815ad25d9d4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:03.888451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27bbe93cd56740ee847c815ad25d9d4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:03.889051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27bbe93cd56740ee847c815ad25d9d4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:03.896174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27bbe93cd56740ee847c815ad25d9d4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:03.908787 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27bbe93cd56740ee847c815ad25d9d4d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:03.959873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1353/2738] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:03 2022] DELETE /identity/v3/users/27bbe93cd56740ee847c815ad25d9d4d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:03.964456 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:03.967983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:03.967983 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:03.984132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:03.985329 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:04.042753 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1386/2739] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:04.052826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:04.054279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:04.110638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b4d7b-83a0-491c-9dce-a0dc5bc81949 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:04.114188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:04.115316 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:04.115594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:04.115868 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:04.124764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:04.126195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:04.136215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1354/2740] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:04.515664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:04.518223 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:04.532254 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548477d-bd2f-47bd-bfb2-dc4c4c3908cd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:04.537645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:04.538279 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:04.538631 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:04.538862 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] PATH_INFO: `/v3/projects/495ddf7c6d30465b8684811f793b001f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:04.539988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=495ddf7c6d30465b8684811f793b001f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:04.542047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:04.554204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:04.665123 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495ddf7c6d30465b8684811f793b001f', 'request_id': 'req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:04.736357 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495ddf7c6d30465b8684811f793b001f', 'request_id': 'req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:04.743615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495ddf7c6d30465b8684811f793b001f', 'request_id': 'req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:04.753924 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] The token cache is being invalidate because project 495ddf7c6d30465b8684811f793b001f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:04.754210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:04.754847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:04.756343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1387/2741] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:04 2022] DELETE /identity/v3/projects/495ddf7c6d30465b8684811f793b001f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:04.760628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:04.770314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee553c-1eb3-4a61-9141-63bda9d7704e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:04.825483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:04.826114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:04.826368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:04.826678 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] PATH_INFO: `/v3/users/63d1b2a8233e4a289cf5e1fdaf865ef9` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:04.827801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d1b2a8233e4a289cf5e1fdaf865ef9)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:04.829722 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:04.937499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:05.042571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d1b2a8233e4a289cf5e1fdaf865ef9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.043254 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d1b2a8233e4a289cf5e1fdaf865ef9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.043979 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d1b2a8233e4a289cf5e1fdaf865ef9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.051410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d1b2a8233e4a289cf5e1fdaf865ef9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.063511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d0f794b3-7db4-4559-9814-420fbc4a1811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d1b2a8233e4a289cf5e1fdaf865ef9'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.142449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1355/2742] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:04 2022] DELETE /identity/v3/users/63d1b2a8233e4a289cf5e1fdaf865ef9 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:05.527945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:05.529487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:05.542799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695af5a3-3a2e-47b4-b2be-a21c620fb1f8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:05.599370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:05.599992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:05.600276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:05.600548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] PATH_INFO: `/v3/projects/07c99aa3dc9f46c2ac8e52d143cd11b9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:05.601646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c99aa3dc9f46c2ac8e52d143cd11b9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:05.603566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:05.731778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c99aa3dc9f46c2ac8e52d143cd11b9', 'request_id': 'req-ddff7746-3572-4a38-95a5-47773741c0d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.805235 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c99aa3dc9f46c2ac8e52d143cd11b9', 'request_id': 'req-ddff7746-3572-4a38-95a5-47773741c0d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.812378 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c99aa3dc9f46c2ac8e52d143cd11b9', 'request_id': 'req-ddff7746-3572-4a38-95a5-47773741c0d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.824526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] The token cache is being invalidate because project 07c99aa3dc9f46c2ac8e52d143cd11b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:05.824824 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.825450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ddff7746-3572-4a38-95a5-47773741c0d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:05.826998 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1388/2743] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:05 2022] DELETE /identity/v3/projects/07c99aa3dc9f46c2ac8e52d143cd11b9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:05.831966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:05.832257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:05.832537 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:05.851732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:05.852941 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:05.911977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1356/2744] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:05.919664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:05.919945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:05.920217 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:05.940035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:05.941251 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:05.949038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1389/2745] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:05.953408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:05.963166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551bc8b-eaa0-4423-80dc-24afb3f83cdc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:05.967714 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:05.968309 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:05.968589 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:05.968864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:05.969507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:25:05.969809 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:05.971440 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:05.983367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1357/2746] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:25:05.987526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:05.997276 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86489df6-a57e-4002-aeca-1ae1c0d3d37d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.001892 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.002467 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.002769 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.003034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.003649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.005504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:06.103742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1390/2747] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:05 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:06.108210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:06.118635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f409e-2c07-4be7-8711-9d9d471e8c35 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.174943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.175705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.175982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.176252 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.176902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.178851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:06.303305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1358/2748] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:06 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:06.307508 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:06.312867 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:06.319083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8b10-b151-48c4-bb33-9e908b95995e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.323777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.324374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.324654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.324928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.325601 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.327231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:06.337016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1391/2749] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:25:06.340843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:06.350917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4246aaf-e8e5-4ad9-9b81-bf220ea53dd5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.355695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513a9c1-3d62-403d-a162-a87891b5a73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.356226 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513a9c1-3d62-403d-a162-a87891b5a73d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.356506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513a9c1-3d62-403d-a162-a87891b5a73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.356781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513a9c1-3d62-403d-a162-a87891b5a73d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.357424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513a9c1-3d62-403d-a162-a87891b5a73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.359060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513a9c1-3d62-403d-a162-a87891b5a73d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:06.370124 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1359/2750] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:06.374151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:06.384928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffac11-35b4-4bbf-b080-d4b6e744af4c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.389461 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.390097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.390376 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.390689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] PATH_INFO: `/v3/projects/54ba8acc51c1403c9fe4d436f204d06b/users/fa3ccfa35e344e9ab63ef144d5d74ff4/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.405630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ba8acc51c1403c9fe4d436f204d06b, user_id=fa3ccfa35e344e9ab63ef144d5d74ff4, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.408341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:06.473389 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1392/2751] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:06 2022] PUT /identity/v3/projects/54ba8acc51c1403c9fe4d436f204d06b/users/fa3ccfa35e344e9ab63ef144d5d74ff4/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:06.485394 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.485684 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.485962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.506401 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:06.514894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] MFA Rules not processed for user `fa3ccfa35e344e9ab63ef144d5d74ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:06.519140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:06.699771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1360/2752] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:06.704199 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:06.765695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b5d0-6358-4e05-83bc-caa89bc6249a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.770288 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.771453 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.771733 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.772006 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.772677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.774589 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:06.880662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1393/2753] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:06 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:06.884800 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:06.895318 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0873dc7-5864-4c4a-83c2-f26cf3bedebf None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:06.949744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:06.950388 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:06.950641 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:06.950907 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:06.951529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:06.953438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:07.040197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1361/2754] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:06 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:25:07.044253 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:07.055475 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a62186-89ef-4968-bb1f-6f48409e69f8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:07.060074 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:07.060667 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:07.060948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:07.061222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:07.061866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:07.063473 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:07.074662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1394/2755] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:07.078601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:07.088370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c5d18-fb2f-4581-b976-283a62c46bb7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:07.092849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:07.093478 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:07.093747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:07.094016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] PATH_INFO: `/v3/projects/3d6f436262524045b7a97e82f798e804/users/fbcec52620a34f09bd7811218bb3804c/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:07.108296 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6f436262524045b7a97e82f798e804, user_id=fbcec52620a34f09bd7811218bb3804c, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:07.110938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:07.170613 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1362/2756] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:07 2022] PUT /identity/v3/projects/3d6f436262524045b7a97e82f798e804/users/fbcec52620a34f09bd7811218bb3804c/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:25:07.174575 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:07.185370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242ee3e-0116-4366-bad1-3898501d6514 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:07.243348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72958ac2-4b0b-4475-b865-dc93153bd9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:07.243949 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72958ac2-4b0b-4475-b865-dc93153bd9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:07.244224 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72958ac2-4b0b-4475-b865-dc93153bd9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:07.244490 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72958ac2-4b0b-4475-b865-dc93153bd9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:07.245168 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72958ac2-4b0b-4475-b865-dc93153bd9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:07.246784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72958ac2-4b0b-4475-b865-dc93153bd9b7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:07.256295 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1395/2757] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:07.260512 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:07.271436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049455d9-53d8-4cfb-83a1-066224ff5f11 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:07.276280 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:07.276916 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:07.277468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:07.277746 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] PATH_INFO: `/v3/projects/3d6f436262524045b7a97e82f798e804/users/fbcec52620a34f09bd7811218bb3804c/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:07.279700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6f436262524045b7a97e82f798e804, user_id=fbcec52620a34f09bd7811218bb3804c, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:07.282308 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:07.293782 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1363/2758] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:07 2022] PUT /identity/v3/projects/3d6f436262524045b7a97e82f798e804/users/fbcec52620a34f09bd7811218bb3804c/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:07.304177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:07.304460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:07.304736 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:07.332346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] MFA Rules not processed for user `fbcec52620a34f09bd7811218bb3804c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:07.334057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:07.360781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:07.403072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1396/2759] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:07.413843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:07.415499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:07.471136 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9e08-7a5b-48f6-8c37-8426a9325793 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:07.474784 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:07.475368 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:07.475639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:07.475906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:07.479849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:07.531441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:07.532786 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:07.541737 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1364/2760] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:10.628884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:10.632870 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:10.695795 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177a27-d0f4-49a8-9e5e-937b2288f768 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:10.702464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:10.703623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:10.704200 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:10.704669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:10.707370 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:10.806386 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:10.808089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:10.815450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:10.817004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:10.820443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aed0c2-bc5b-4eb1-8e78-c7bad59ae9c6 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:10.824487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:10.825365 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:10.825741 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:10.826118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:10.828127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:10.842395 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1397/2761] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:10.842620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:10.844166 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:10.856231 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1365/2762] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:25:12.862948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:12.866835 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:12.869346 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:12.873049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1029f-e481-42e7-89a5-769a061b03e9 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:12.877336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:12.877974 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:12.878731 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:12.879014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:12.883925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b7eac-5dde-495d-be71-3cbb020bbb27 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:12.888790 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:12.889730 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:12.890225 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:12.890340 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:12.890621 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:12.891596 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:12.892850 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:12.900645 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1398/2763] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:12.906603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:12.908189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201843-15f4-4e4f-a666-85ccabaa264f service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:12.919998 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1366/2764] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:18.623661 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:18.652893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:18.687567 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98fdf6-8b34-40e2-ad2f-298277e818d3 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:18.692234 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:18.692859 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:18.693617 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:18.694007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] PATH_INFO: `/v3/users/fa3ccfa35e344e9ab63ef144d5d74ff4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:18.695320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3ccfa35e344e9ab63ef144d5d74ff4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:18.697209 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:18.871427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3ccfa35e344e9ab63ef144d5d74ff4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:18.871997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3ccfa35e344e9ab63ef144d5d74ff4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:18.872658 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3ccfa35e344e9ab63ef144d5d74ff4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:18.880231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3ccfa35e344e9ab63ef144d5d74ff4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:18.898414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3ccfa35e344e9ab63ef144d5d74ff4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:18.986618 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1399/2765] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:18 2022] DELETE /identity/v3/users/fa3ccfa35e344e9ab63ef144d5d74ff4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:18.988314 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:18.988811 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:18.988913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:19.014471 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:19.015957 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:19.094026 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1367/2766] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:19.110169 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:19.111711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:19.185640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82686f03-eff8-4d40-b445-27c30af60b14 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:19.189749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:19.190348 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:19.190650 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:19.190927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:19.201564 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:19.203045 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:19.215366 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1400/2767] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:19.734044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:19.738903 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:19.750956 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df13bd2-da71-41a7-9646-48068b6f2fd9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:19.759514 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:19.760705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:19.760995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:19.761515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] PATH_INFO: `/v3/projects/54ba8acc51c1403c9fe4d436f204d06b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:19.762431 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ba8acc51c1403c9fe4d436f204d06b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:19.764682 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:19.864726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ba8acc51c1403c9fe4d436f204d06b', 'request_id': 'req-4d3f5503-eb69-4480-9d4c-a3bdd8938189'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:19.871435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ba8acc51c1403c9fe4d436f204d06b', 'request_id': 'req-4d3f5503-eb69-4480-9d4c-a3bdd8938189'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:19.877880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ba8acc51c1403c9fe4d436f204d06b', 'request_id': 'req-4d3f5503-eb69-4480-9d4c-a3bdd8938189'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:19.930060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] The token cache is being invalidate because project 54ba8acc51c1403c9fe4d436f204d06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:19.930287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:19.930904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:19.932400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1368/2768] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:19 2022] DELETE /identity/v3/projects/54ba8acc51c1403c9fe4d436f204d06b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:19.936546 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:19.946372 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6a77d-350a-473b-accd-1ab892ed1dca service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:19.990578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:20.002884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:20.003509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:20.003786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:20.004060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] PATH_INFO: `/v3/users/fbcec52620a34f09bd7811218bb3804c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:20.005176 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcec52620a34f09bd7811218bb3804c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:20.007140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:20.213271 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcec52620a34f09bd7811218bb3804c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.213910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcec52620a34f09bd7811218bb3804c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.214648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcec52620a34f09bd7811218bb3804c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.221903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcec52620a34f09bd7811218bb3804c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.233872 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6db0f5b5-9ee7-4674-bde8-2061c93f5a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcec52620a34f09bd7811218bb3804c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.325539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1401/2769] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:19 2022] DELETE /identity/v3/users/fbcec52620a34f09bd7811218bb3804c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:25:20.767135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:20.769240 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:20.790436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:20.797501 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f5503-eb69-4480-9d4c-a3bdd8938189 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:20.856068 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:20.856700 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:20.856978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:20.857250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] PATH_INFO: `/v3/projects/3d6f436262524045b7a97e82f798e804` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:20.858358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6f436262524045b7a97e82f798e804)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:20.860322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:20.964154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6f436262524045b7a97e82f798e804', 'request_id': 'req-46f016e2-856e-4ce6-a053-aea2b7241574'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.971355 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6f436262524045b7a97e82f798e804', 'request_id': 'req-46f016e2-856e-4ce6-a053-aea2b7241574'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:20.978454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6f436262524045b7a97e82f798e804', 'request_id': 'req-46f016e2-856e-4ce6-a053-aea2b7241574'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:21.095590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] The token cache is being invalidate because project 3d6f436262524045b7a97e82f798e804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:21.095590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:21.095590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-46f016e2-856e-4ce6-a053-aea2b7241574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:21.095590 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1369/2770] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:20 2022] DELETE /identity/v3/projects/3d6f436262524045b7a97e82f798e804 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:21.101644 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.102135 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.102252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.123847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:21.125071 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:21.201438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1402/2771] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:21.209780 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.211220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.213006 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.235091 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:21.236300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:21.245599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1370/2772] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:21.249961 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.260412 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68f50-f9af-4116-b23e-1ec86c0de04d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:21.267544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:21.267544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.267544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.267544 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.269415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:25:21.269415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:21.269415 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:21.289057 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1403/2773] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:25:21.293412 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.303344 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f431659-f5c5-4282-b4d8-5858ecdd4bdf None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:21.308111 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:21.308713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.308996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.309263 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.309896 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:21.312038 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:21.415105 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1371/2774] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:21 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:21.419142 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.429671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc386-0000-4f14-b4b7-705ac3e3148b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:21.490134 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:21.501488 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:21.502221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.502585 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.502930 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.503586 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:21.505596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:21.570432 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1404/2775] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:21 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:21.572251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.606053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06963f-a9b6-4645-a47c-c9df14bf1539 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:21.617548 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:21.618208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.620826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.621125 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.621793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:21.623820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:21.637027 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1372/2776] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:21.651806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.656810 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af102d9f-a816-422c-8934-ca7e1af9e311 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:21.671649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75a93-ba0f-455e-85b8-dcdd9f994017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:21.672428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a93-ba0f-455e-85b8-dcdd9f994017 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.672781 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a93-ba0f-455e-85b8-dcdd9f994017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.673128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a93-ba0f-455e-85b8-dcdd9f994017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.674049 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff75a93-ba0f-455e-85b8-dcdd9f994017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:21.680495 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff75a93-ba0f-455e-85b8-dcdd9f994017 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:21.697221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1405/2777] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:21.701369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.712826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc686e-72c4-420c-950e-82e8b7bd6d05 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:21.717457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:21.717972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.718389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.718500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] PATH_INFO: `/v3/projects/2e211a112f5449c4be56bc41d2a2f917/users/1e641dca220b4f3fa0bcdbab74d19b27/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.733969 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e211a112f5449c4be56bc41d2a2f917, user_id=1e641dca220b4f3fa0bcdbab74d19b27, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:21.736762 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:21.823165 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1373/2778] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:21 2022] PUT /identity/v3/projects/2e211a112f5449c4be56bc41d2a2f917/users/1e641dca220b4f3fa0bcdbab74d19b27/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:21.836416 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:21.836711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:21.836989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:21.873851 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] MFA Rules not processed for user `1e641dca220b4f3fa0bcdbab74d19b27`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:21.875634 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:21.949707 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1406/2779] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:21.962256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:21.964102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:22.000567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:22.028755 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1d0fd-d42a-4579-9f25-459ac5b53b76 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:22.032670 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fc5d3-deda-41c1-919b-5a9cca1f0faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:22.033709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fc5d3-deda-41c1-919b-5a9cca1f0faf service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:22.034205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fc5d3-deda-41c1-919b-5a9cca1f0faf service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:22.034817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fc5d3-deda-41c1-919b-5a9cca1f0faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:22.045337 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fc5d3-deda-41c1-919b-5a9cca1f0faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:22.046843 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fc5d3-deda-41c1-919b-5a9cca1f0faf service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:22.057671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1374/2780] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:22.083714 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:22.157119 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1a79-cd75-4a80-8654-f2c9ab179f22 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:22.161849 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:22.162471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:22.162779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:22.163053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] PATH_INFO: `/v3/users/1e641dca220b4f3fa0bcdbab74d19b27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:22.164262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e641dca220b4f3fa0bcdbab74d19b27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:22.166158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:22.321885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e641dca220b4f3fa0bcdbab74d19b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:22.322536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e641dca220b4f3fa0bcdbab74d19b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:22.323165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e641dca220b4f3fa0bcdbab74d19b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:22.330374 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e641dca220b4f3fa0bcdbab74d19b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:22.335561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:22.342421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e641dca220b4f3fa0bcdbab74d19b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:22.417096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1407/2781] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:22 2022] DELETE /identity/v3/users/1e641dca220b4f3fa0bcdbab74d19b27 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:22.421773 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:22.422053 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:22.422421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:22.441535 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:22.442748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:22.502077 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1375/2782] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:22.511845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:22.513256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:22.573239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2bcc8-41aa-4706-8bde-d0bbbc1232d3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:22.577026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc1666-7b40-42c8-9447-06404068cdc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:22.577630 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1666-7b40-42c8-9447-06404068cdc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:22.577910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1666-7b40-42c8-9447-06404068cdc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:22.578184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1666-7b40-42c8-9447-06404068cdc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:22.587536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc1666-7b40-42c8-9447-06404068cdc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:22.588938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc1666-7b40-42c8-9447-06404068cdc6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:22.599014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1408/2783] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:25:23.018058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.019643 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:23.032416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82941e6-fa18-443e-a234-137d10d38946 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.040294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.041395 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.041687 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.042130 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] PATH_INFO: `/v3/projects/2e211a112f5449c4be56bc41d2a2f917` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.043540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e211a112f5449c4be56bc41d2a2f917)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.045619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.155861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e211a112f5449c4be56bc41d2a2f917', 'request_id': 'req-16d035ff-e668-4f49-9eff-08f2808e0f43'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:23.161983 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e211a112f5449c4be56bc41d2a2f917', 'request_id': 'req-16d035ff-e668-4f49-9eff-08f2808e0f43'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:23.167792 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e211a112f5449c4be56bc41d2a2f917', 'request_id': 'req-16d035ff-e668-4f49-9eff-08f2808e0f43'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:23.233238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] The token cache is being invalidate because project 2e211a112f5449c4be56bc41d2a2f917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:23.233524 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:23.234172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-16d035ff-e668-4f49-9eff-08f2808e0f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:23.235697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1376/2784] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:23 2022] DELETE /identity/v3/projects/2e211a112f5449c4be56bc41d2a2f917 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:23.240979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.241381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.241502 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.261921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:23.263241 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:23.324663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1409/2785] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:23.332489 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.332776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.333092 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.353329 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:23.354571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:23.362385 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1377/2786] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:25:23.366591 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.376177 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7a927-feb1-4bec-b56c-cfe6968c05e9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.381206 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.381798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.382080 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.382351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.383028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:25:23.383336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.384902 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.397711 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1410/2787] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:25:23.402761 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.412099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb930-53d2-4345-a414-c1d28ff6d4d3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.416628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.417216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.417492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.417759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.418383 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.420315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.485844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1378/2788] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:23 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:23.490687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.500977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdd31c-ee45-4f00-b719-53dcc29da023 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.556860 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.557464 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.557742 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.558067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.558750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.560649 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.622971 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:23.627774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1411/2789] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:23 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:23.632608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.642711 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4449761-6e6c-4646-8da9-78d9ab53ba31 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.647593 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.648191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.648465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.648734 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.649410 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.651060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.661336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1379/2790] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:23.665397 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.676321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f8bac-6162-4e70-8147-da10077f25c9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.681566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c3ce3-3e99-4895-958e-e38f8d04dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.682932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c3ce3-3e99-4895-958e-e38f8d04dd72 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.682932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c3ce3-3e99-4895-958e-e38f8d04dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.682932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c3ce3-3e99-4895-958e-e38f8d04dd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.683392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c3ce3-3e99-4895-958e-e38f8d04dd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.684964 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c3ce3-3e99-4895-958e-e38f8d04dd72 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.696477 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1412/2791] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:23.700495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.710442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e2e83-a7aa-4993-b637-99d2418d1c71 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.715300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.720197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.720197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.720197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] PATH_INFO: `/v3/projects/67ebd705b0d94555bbc24018ac4b9d0f/users/5076bee4033c4fa8837c3273a2e53a7a/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.728772 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:23.731853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ebd705b0d94555bbc24018ac4b9d0f, user_id=5076bee4033c4fa8837c3273a2e53a7a, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.734607 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.793744 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1380/2792] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:23 2022] PUT /identity/v3/projects/67ebd705b0d94555bbc24018ac4b9d0f/users/5076bee4033c4fa8837c3273a2e53a7a/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:23.799038 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.799321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.799596 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.820101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:23.821621 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:23.884543 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1413/2793] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:23.894255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:23.896034 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:23.952864 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e050ec-61a2-491f-9678-ab1aeb3f7099 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:23.956477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8bc16-93bc-4cd1-b200-a516912262e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:23.957082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8bc16-93bc-4cd1-b200-a516912262e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:23.957362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8bc16-93bc-4cd1-b200-a516912262e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:23.957637 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8bc16-93bc-4cd1-b200-a516912262e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:23.968218 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8bc16-93bc-4cd1-b200-a516912262e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:23.969671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8bc16-93bc-4cd1-b200-a516912262e6 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:23.980571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1381/2794] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:26.149460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:26.151077 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:26.164451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dfb99-e2c1-4148-90f2-0683f87ae78e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:26.171341 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:26.172024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:26.172307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:26.172806 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:26.174994 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:26.190042 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:26.191561 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:26.203774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1414/2795] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:31.075373 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.076936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.077397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.136441 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] MFA Rules not processed for user `5076bee4033c4fa8837c3273a2e53a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:31.137801 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:31.244578 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.246580 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:31.285357 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1382/2796] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:31.289708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.303334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e225b-7a8d-4c89-861b-9442b62b6575 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.313545 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.314456 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.314913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.315332 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.316167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.318910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:31.417881 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1383/2797] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:25:31 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:25:31.419631 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.422039 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40909b4a-b449-44e5-8941-9646f15a3a13 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.496294 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.496910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.497190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.497463 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.498362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf2ee-4237-4931-ac7d-43f309dc28c3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.499252 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:31.503302 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.504153 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.504437 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.504712 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.505483 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.507449 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:31.510623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.512195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:31.516706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:31.534653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1415/2798] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:25:31.619420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1384/2799] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:25:31 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:25:31.623178 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.638280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b47c8-3194-4bea-99be-93ce05e94749 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.643309 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.645897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.645897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.645897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.645897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.647190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:31.649833 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:31.662904 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1416/2800] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:25:31.664177 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.694233 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db71b6-b697-4182-98aa-fcbf1429903d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.701320 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.701980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.709086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.709086 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] PATH_INFO: `/v3/projects/a43589ded36843fd8bb9c7046d9614df/users/fbb00d3e463e45e586b274af0de1f4d5/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.720384 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43589ded36843fd8bb9c7046d9614df, user_id=fbb00d3e463e45e586b274af0de1f4d5, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.723513 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:31.836595 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1385/2801] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:31 2022] PUT /identity/v3/projects/a43589ded36843fd8bb9c7046d9614df/users/fbb00d3e463e45e586b274af0de1f4d5/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:31.840681 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.854255 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86b35e-abd9-4774-b6a3-e92e0c9d50d5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.935654 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5705dbe-6402-450a-b6fb-1ae59d3bdb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.936278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5705dbe-6402-450a-b6fb-1ae59d3bdb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.936558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5705dbe-6402-450a-b6fb-1ae59d3bdb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.936831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5705dbe-6402-450a-b6fb-1ae59d3bdb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.937510 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5705dbe-6402-450a-b6fb-1ae59d3bdb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.939131 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5705dbe-6402-450a-b6fb-1ae59d3bdb72 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:31.950265 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1417/2802] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:25:31.954474 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:31.966610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262e8aa-62da-48fb-9671-c5349264f3f5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:31.975271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:31.978624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:31.978624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:31.978624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] PATH_INFO: `/v3/projects/a43589ded36843fd8bb9c7046d9614df/users/fbb00d3e463e45e586b274af0de1f4d5/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:31.978624 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43589ded36843fd8bb9c7046d9614df, user_id=fbb00d3e463e45e586b274af0de1f4d5, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:31.982844 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:32.001880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1386/2803] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:25:31 2022] PUT /identity/v3/projects/a43589ded36843fd8bb9c7046d9614df/users/fbb00d3e463e45e586b274af0de1f4d5/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:34.802743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:34.803773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:34.804301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:34.830439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] MFA Rules not processed for user `c639d3e06c4c4fcc834979684b00af6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:34.831710 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:34.906680 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1418/2804] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:36.149085 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:36.151007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:36.257561 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d541b3-b0c4-4730-bb3f-1101d7a97060 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:36.265351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:36.265914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:36.266251 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:36.266691 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] PATH_INFO: `/v3/users/f9db5f73350c460f99e4ab5cdbe6907a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:36.267822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9db5f73350c460f99e4ab5cdbe6907a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:36.270014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:36.456882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9db5f73350c460f99e4ab5cdbe6907a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:36.457653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9db5f73350c460f99e4ab5cdbe6907a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:36.458874 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9db5f73350c460f99e4ab5cdbe6907a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:36.466820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9db5f73350c460f99e4ab5cdbe6907a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:36.482974 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9db5f73350c460f99e4ab5cdbe6907a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:36.560442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1387/2805] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:36 2022] DELETE /identity/v3/users/f9db5f73350c460f99e4ab5cdbe6907a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:36.739999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:36.837958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422618-d826-4b25-9e9f-f6c5e2c34ae5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:36.898036 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:36.924717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:36.933115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:36.933257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:36.933257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:36.933257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:36.933257 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:36.933625 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:36.947047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:36.950582 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:36.967101 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1419/2806] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:36.985110 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:37.030237 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f46f4-e47d-433c-9f40-3007ac208222 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:37.040121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:37.040121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:37.040121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:37.040121 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] PATH_INFO: `/v3/projects/7847d148d828478b8124c8d08c8c582f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:37.043298 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7847d148d828478b8124c8d08c8c582f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:37.045764 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:37.171207 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7847d148d828478b8124c8d08c8c582f', 'request_id': 'req-f30f511d-b2b4-4696-bd2e-83a33ec21385'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.181493 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7847d148d828478b8124c8d08c8c582f', 'request_id': 'req-f30f511d-b2b4-4696-bd2e-83a33ec21385'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.188551 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7847d148d828478b8124c8d08c8c582f', 'request_id': 'req-f30f511d-b2b4-4696-bd2e-83a33ec21385'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.277831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] The token cache is being invalidate because project 7847d148d828478b8124c8d08c8c582f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:37.278258 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.279477 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.283686 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1388/2807] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:36 2022] DELETE /identity/v3/projects/7847d148d828478b8124c8d08c8c582f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:25:37.285445 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:37.300825 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b9eae-7102-4d36-ba26-626df9be8d9b service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:37.401454 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:37.402602 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:37.402992 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:37.403273 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] PATH_INFO: `/v3/users/4511c7c8d2e5433aa1c496c9193ecb32` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:37.404468 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4511c7c8d2e5433aa1c496c9193ecb32)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:37.407695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:37.604937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4511c7c8d2e5433aa1c496c9193ecb32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.605937 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4511c7c8d2e5433aa1c496c9193ecb32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.607351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4511c7c8d2e5433aa1c496c9193ecb32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.623487 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4511c7c8d2e5433aa1c496c9193ecb32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.643072 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e79994fc-bcbd-40e4-867c-9bbef6701cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4511c7c8d2e5433aa1c496c9193ecb32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:37.726832 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1420/2808] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:25:37 2022] DELETE /identity/v3/users/4511c7c8d2e5433aa1c496c9193ecb32 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:38.081754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:38.081754 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:38.104118 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f511d-b2b4-4696-bd2e-83a33ec21385 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:38.224901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:38.226292 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:38.226781 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:38.227250 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] PATH_INFO: `/v3/projects/f07be67668fc4a95a968791b721daec2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:38.228826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07be67668fc4a95a968791b721daec2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:38.230993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:38.396756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07be67668fc4a95a968791b721daec2', 'request_id': 'req-7325ee1e-14f7-49ea-b2bd-3381d85749b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:38.405673 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07be67668fc4a95a968791b721daec2', 'request_id': 'req-7325ee1e-14f7-49ea-b2bd-3381d85749b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:38.414036 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07be67668fc4a95a968791b721daec2', 'request_id': 'req-7325ee1e-14f7-49ea-b2bd-3381d85749b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:38.417519 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:38.497747 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] The token cache is being invalidate because project f07be67668fc4a95a968791b721daec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:25:38.498261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:38.499127 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:25:38.501364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1389/2809] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:25:38 2022] DELETE /identity/v3/projects/f07be67668fc4a95a968791b721daec2 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:25:39.108141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:39.108687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:39.108687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:39.143427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] MFA Rules not processed for user `fbb00d3e463e45e586b274af0de1f4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:39.144695 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:39.194484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:39.256972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1421/2810] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:39.268867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:39.270921 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:39.367389 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325ee1e-14f7-49ea-b2bd-3381d85749b4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:39.374425 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:39.377958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:39.379910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:39.379910 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:39.380622 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:39.461853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:39.463542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbb103-d239-4eb8-9c5b-0cdce867c0e6 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:39.474783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1390/2811] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:40.375305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:40.376764 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:40.439124 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450c067-fc30-4f06-b6ef-e0c693abd9ba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:40.443301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:40.443906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:40.444181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:40.444450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:40.446246 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:40.457301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:40.459663 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:40.471948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1422/2812] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:42.178528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:42.179056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:42.180912 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:42.201421 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] MFA Rules not processed for user `5076bee4033c4fa8837c3273a2e53a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:25:42.202699 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:42.215652 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1391/2813] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:25:48.390442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:48.391975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:48.407999 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c1e3d-a116-44d9-9cfe-5c3e5f1b87d5 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:48.413677 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:48.414275 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:48.414588 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:48.414863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:48.416652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:48.429655 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:48.431277 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:48.443094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1423/2814] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:52.192695 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=1,keystone:SELECT=61 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:25:52.309010 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:52.322715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe2fd0-1e45-413d-856d-dc644c94d770 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:52.328424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:52.329029 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:52.329422 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:52.329698 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:52.331861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:52.344336 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:52.345999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:52.358248 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1392/2815] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:25:58.139988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:25:58.141381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:58.153576 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536267-ad4f-4fce-8587-7446fddc8e59 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:25:58.157166 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:25:58.157777 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:25:58.158054 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:25:58.158327 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:25:58.160035 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:25:58.170094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:25:58.171536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:25:58.179537 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1424/2816] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:25:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:26:02.353106 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:08.175886 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=49 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:16.785305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:16.795216 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ccc99-205b-4bbc-997f-8401bdf76c5b service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:16.798790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:16.799376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:16.799647 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:16.799913 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:16.808525 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:16.809928 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:16.819279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1393/2817] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:26:26.814954 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:50.619587 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:50.628997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902b25-9eb4-463a-ae3f-871ee6770acb service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:50.632762 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:50.633350 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:50.633629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:50.633903 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:50.635640 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:50.697719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:50.699160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:50.708689 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1425/2818] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:26:54.265021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:54.266481 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:54.340873 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408c737-1d98-4e3e-8289-a9f1cc9c882e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:54.345917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:54.346507 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:54.346803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:54.347070 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:54.348819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:54.361438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:54.365664 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:54.366208 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:54.367385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:54.377306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1394/2819] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:26:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:26:54.383599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b5e9-ab63-418d-bd9b-29eaf81cb63f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:54.388418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:54.389046 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:54.389448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:54.389597 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] PATH_INFO: `/v3/users/c639d3e06c4c4fcc834979684b00af6c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:54.390749 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c639d3e06c4c4fcc834979684b00af6c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:54.392641 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:54.547914 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c639d3e06c4c4fcc834979684b00af6c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:54.548557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c639d3e06c4c4fcc834979684b00af6c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:54.549196 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c639d3e06c4c4fcc834979684b00af6c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:54.556403 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c639d3e06c4c4fcc834979684b00af6c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:54.569247 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a26e141c-5350-478d-99cc-a38666564aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c639d3e06c4c4fcc834979684b00af6c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:54.642518 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1426/2820] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:26:54 2022] DELETE /identity/v3/users/c639d3e06c4c4fcc834979684b00af6c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:26:54.856150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:54.867671 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aa34f-5117-4a4c-b3d6-dda48f15f8cf service placement] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:54.931605 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:54.932150 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:54.932430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:54.932702 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] PATH_INFO: `/v3/projects/56c66a304fdf4e9eba6ca0b7a15f492b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:54.933820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c66a304fdf4e9eba6ca0b7a15f492b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:54.935759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.014141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c66a304fdf4e9eba6ca0b7a15f492b', 'request_id': 'req-52122206-e821-4ee6-8781-35d7c9a0624b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:55.020196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c66a304fdf4e9eba6ca0b7a15f492b', 'request_id': 'req-52122206-e821-4ee6-8781-35d7c9a0624b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:55.025721 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c66a304fdf4e9eba6ca0b7a15f492b', 'request_id': 'req-52122206-e821-4ee6-8781-35d7c9a0624b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:55.106182 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] The token cache is being invalidate because project 56c66a304fdf4e9eba6ca0b7a15f492b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:26:55.106504 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:55.107646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52122206-e821-4ee6-8781-35d7c9a0624b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:26:55.109175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1395/2821] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:26:54 2022] DELETE /identity/v3/projects/56c66a304fdf4e9eba6ca0b7a15f492b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:26:55.114759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.114759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.114979 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.135763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:26:55.136989 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:55.199112 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1427/2822] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:26:55.207205 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.207490 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.207765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.227391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:26:55.228588 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:55.237255 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1396/2823] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:26:55.241471 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.251802 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de358970-af09-4bd3-9947-b73d3ac5a30a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.256346 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.256934 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.257212 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.257484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.258148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:26:55.258448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.260044 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.261478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:55.272132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1428/2824] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:26:55.278197 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.287674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b03aca-d6f5-4177-ad59-5f47b0aea02e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.292244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.292831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.293109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.293380 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.294013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.296430 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.307530 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:55.398885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1397/2825] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/projects => generated 393 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:26:55.403304 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.413132 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc860b5-b276-4afa-96f6-45274f181b3a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.470960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.471484 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.471759 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.472024 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.472648 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.474527 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.552095 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1429/2826] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:26:55.556310 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.566876 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7f01c-b5ef-4cfd-92ba-22b7c554f46d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.571499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.572090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.572369 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.572690 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.573331 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.575009 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.585013 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1398/2827] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:26:55.589021 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.599776 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4560a4-d3b1-4c05-b039-374d18dc0f08 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.604368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51b0c-16a7-4a00-a9d4-b603fceeffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.604958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51b0c-16a7-4a00-a9d4-b603fceeffcf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.605237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51b0c-16a7-4a00-a9d4-b603fceeffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.605509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51b0c-16a7-4a00-a9d4-b603fceeffcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.606145 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51b0c-16a7-4a00-a9d4-b603fceeffcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.607757 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51b0c-16a7-4a00-a9d4-b603fceeffcf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.618881 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1430/2828] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:26:55.622898 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.632705 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f64abc-13e2-46c4-9250-0317542a6a5b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.637379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.638021 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.638299 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.638600 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] PATH_INFO: `/v3/projects/b570b127b1074e6cb54ccf4725d64bc2/users/070af786c6694ffe997df1b228c0e340/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.653000 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b570b127b1074e6cb54ccf4725d64bc2, user_id=070af786c6694ffe997df1b228c0e340, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.656179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.705555 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1399/2829] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:26:55 2022] PUT /identity/v3/projects/b570b127b1074e6cb54ccf4725d64bc2/users/070af786c6694ffe997df1b228c0e340/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:26:55.716652 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.716940 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.717214 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.746307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] MFA Rules not processed for user `070af786c6694ffe997df1b228c0e340`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:26:55.747455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:55.811428 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1431/2830] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:26:55.815958 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.877776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108d69b-98c3-4b30-a677-a98798c24ad7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:55.882321 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:55.882942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:55.883215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:55.883484 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:55.884099 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:55.885977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:55.952577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1400/2831] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:26:55.956784 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:55.967442 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271263c-6b3f-4a0e-8292-9618da3fab86 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.023030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.023552 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.023827 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.024094 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.024722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.026885 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.035623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:56.128955 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1432/2832] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:26:55 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:26:56.132929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:56.145417 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665bac5-6405-4874-9ea8-376d908b62ed admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.150108 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.150851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.151140 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.151416 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.152313 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.153901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.164639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1401/2833] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:26:56.168754 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:56.181568 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bad67-70d2-441a-8291-7085dcb36db2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.187089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.187743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.188090 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.188360 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] PATH_INFO: `/v3/projects/45229ba8a9ef460090c0648a262dcefd/users/2c56e032313d41cebef9fba64fdc3c76/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.203497 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45229ba8a9ef460090c0648a262dcefd, user_id=2c56e032313d41cebef9fba64fdc3c76, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.206140 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.254450 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1433/2834] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:26:56 2022] PUT /identity/v3/projects/45229ba8a9ef460090c0648a262dcefd/users/2c56e032313d41cebef9fba64fdc3c76/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:26:56.259639 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:56.268982 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fdac26-cd93-4ef1-8905-6b67556eb67f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.324202 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e40d28-452f-4c5b-84b6-7f7e00985107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.324848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40d28-452f-4c5b-84b6-7f7e00985107 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.325014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40d28-452f-4c5b-84b6-7f7e00985107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.325287 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40d28-452f-4c5b-84b6-7f7e00985107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.325924 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e40d28-452f-4c5b-84b6-7f7e00985107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.327614 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e40d28-452f-4c5b-84b6-7f7e00985107 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.337573 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1402/2835] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:26:56.341725 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:56.354789 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3b176-223f-4e34-a58e-64b7be73f344 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.360022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.360662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.360935 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.361204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] PATH_INFO: `/v3/projects/45229ba8a9ef460090c0648a262dcefd/users/2c56e032313d41cebef9fba64fdc3c76/roles/399ebd44d9c74dcc9954c0828ffd63c6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.363122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45229ba8a9ef460090c0648a262dcefd, user_id=2c56e032313d41cebef9fba64fdc3c76, role_id=399ebd44d9c74dcc9954c0828ffd63c6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.365724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.375890 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1434/2836] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:26:56 2022] PUT /identity/v3/projects/45229ba8a9ef460090c0648a262dcefd/users/2c56e032313d41cebef9fba64fdc3c76/roles/399ebd44d9c74dcc9954c0828ffd63c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:26:56.386061 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64386344-872d-4e5a-a718-3285cda70395 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.386348 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64386344-872d-4e5a-a718-3285cda70395 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.386665 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64386344-872d-4e5a-a718-3285cda70395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.415256 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-64386344-872d-4e5a-a718-3285cda70395 None None] MFA Rules not processed for user `2c56e032313d41cebef9fba64fdc3c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:26:56.416463 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-64386344-872d-4e5a-a718-3285cda70395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:56.429294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:26:56.487333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1403/2837] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:26:56.498425 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:56.500408 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:56.557642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ce710-fe84-459c-957d-f05f5b30ab1b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.561311 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.561900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.562175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.562443 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.571548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.573083 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.583069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1435/2838] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:26:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:26:56.628992 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64386344-872d-4e5a-a718-3285cda70395 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:26:56.630376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-64386344-872d-4e5a-a718-3285cda70395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:26:56.642586 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64386344-872d-4e5a-a718-3285cda70395 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:26:56.646089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3d13-536e-4190-8c2d-5d451f8a05f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:26:56.646708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3d13-536e-4190-8c2d-5d451f8a05f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:26:56.646980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3d13-536e-4190-8c2d-5d451f8a05f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:26:56.647244 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3d13-536e-4190-8c2d-5d451f8a05f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:26:56.701011 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3d13-536e-4190-8c2d-5d451f8a05f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:26:56.703084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3d13-536e-4190-8c2d-5d451f8a05f2 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:26:56.712723 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1404/2839] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:26:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:01.201394 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:01.262569 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4ceb2-e7eb-4bba-8724-dd180a6c22fa service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:01.267690 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:01.268301 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:01.268581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:01.268846 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] PATH_INFO: `/v3/users/070af786c6694ffe997df1b228c0e340` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:01.269946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=070af786c6694ffe997df1b228c0e340)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:01.271878 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:01.445281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070af786c6694ffe997df1b228c0e340'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:01.445921 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070af786c6694ffe997df1b228c0e340'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:01.446524 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070af786c6694ffe997df1b228c0e340'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:01.453592 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070af786c6694ffe997df1b228c0e340'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:01.455498 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:01.465727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070af786c6694ffe997df1b228c0e340'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:01.534922 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1436/2840] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:27:01 2022] DELETE /identity/v3/users/070af786c6694ffe997df1b228c0e340 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:01.539601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:01.540084 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:01.540209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:01.559427 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:01.560611 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:01.619192 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1405/2841] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:01.629148 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:01.630516 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:01.685957 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400508a8-76eb-4b28-a1bd-672d46ff6897 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:01.690343 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:01.690981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:01.691262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:01.691538 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:01.700612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:01.701958 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:01.711280 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1437/2842] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:02.147132 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:02.148628 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:02.162511 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396b769-b664-4d67-9935-6c5e16a5cb7d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:02.169035 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:02.169669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:02.169942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:02.170209 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] PATH_INFO: `/v3/projects/b570b127b1074e6cb54ccf4725d64bc2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:02.171402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b570b127b1074e6cb54ccf4725d64bc2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:02.174093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:02.274935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:02.278028 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b570b127b1074e6cb54ccf4725d64bc2', 'request_id': 'req-21c2c05f-c430-4e6a-a001-b3c795cdb721'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.284978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b570b127b1074e6cb54ccf4725d64bc2', 'request_id': 'req-21c2c05f-c430-4e6a-a001-b3c795cdb721'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.290666 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b570b127b1074e6cb54ccf4725d64bc2', 'request_id': 'req-21c2c05f-c430-4e6a-a001-b3c795cdb721'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.362052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] The token cache is being invalidate because project b570b127b1074e6cb54ccf4725d64bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:27:02.362339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.362972 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.364451 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1406/2843] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:27:02 2022] DELETE /identity/v3/projects/b570b127b1074e6cb54ccf4725d64bc2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:02.368512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:02.378458 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef231f4-81ee-4cb0-ad43-6ccef6725611 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:02.436018 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:02.436635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:02.436909 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:02.439040 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] PATH_INFO: `/v3/users/2c56e032313d41cebef9fba64fdc3c76` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:02.440210 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c56e032313d41cebef9fba64fdc3c76)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:02.442089 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:02.610981 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c56e032313d41cebef9fba64fdc3c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.611643 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c56e032313d41cebef9fba64fdc3c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.612221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c56e032313d41cebef9fba64fdc3c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.619194 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c56e032313d41cebef9fba64fdc3c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.631313 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b8e3425d-29cd-470c-a4da-2c47aa667990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c56e032313d41cebef9fba64fdc3c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:02.689158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:02.694228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1438/2844] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:27:02 2022] DELETE /identity/v3/users/2c56e032313d41cebef9fba64fdc3c76 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:27:03.118750 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.121397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:03.134848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2c05f-c430-4e6a-a001-b3c795cdb721 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.197214 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.197840 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.198115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.198387 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] PATH_INFO: `/v3/projects/45229ba8a9ef460090c0648a262dcefd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.199546 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45229ba8a9ef460090c0648a262dcefd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.201436 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.311176 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45229ba8a9ef460090c0648a262dcefd', 'request_id': 'req-ec6bb569-15b8-46d4-81be-2d33f3833e4b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:03.317866 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45229ba8a9ef460090c0648a262dcefd', 'request_id': 'req-ec6bb569-15b8-46d4-81be-2d33f3833e4b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:03.323358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45229ba8a9ef460090c0648a262dcefd', 'request_id': 'req-ec6bb569-15b8-46d4-81be-2d33f3833e4b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:03.389994 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] The token cache is being invalidate because project 45229ba8a9ef460090c0648a262dcefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:27:03.390852 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:03.396849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ec6bb569-15b8-46d4-81be-2d33f3833e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:03.396849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1407/2845] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:27:03 2022] DELETE /identity/v3/projects/45229ba8a9ef460090c0648a262dcefd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:03.398399 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.398910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.399047 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.418614 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:03.419814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:03.478152 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1439/2846] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:03.487156 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.487443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.487724 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.508433 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:03.509650 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:03.518339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1408/2847] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:03.522692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.532660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf4cf3-0689-489d-9585-234567cc01ce None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.537227 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.537821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.538143 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.538414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.539099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:27:03.539401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.540963 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.552440 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1440/2848] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:27:03.556719 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.566351 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acbedb-a640-40f8-bf9e-b6aad2a6250e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.571015 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.571601 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.571880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.572152 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.572771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.574715 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.660967 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1409/2849] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:27:03 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:27:03.665009 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.675608 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2409e2-4e85-4632-809b-31c106f4671e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.731623 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.732222 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.732504 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.732780 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.733431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.735856 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.802747 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1441/2850] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:27:03 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:27:03.806735 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.817544 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d8431-4a57-4445-b2aa-0eb7dfce17e0 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.822234 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.822855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.823135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.823408 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.824081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.825668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.835933 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1410/2851] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:03.839604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.850272 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892cbeb-2104-4987-9926-8efd22805c3f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.854928 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd10f9-871f-40c8-b751-6e339b0878ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.855511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd10f9-871f-40c8-b751-6e339b0878ea admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.855786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd10f9-871f-40c8-b751-6e339b0878ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.856081 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd10f9-871f-40c8-b751-6e339b0878ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.856715 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd10f9-871f-40c8-b751-6e339b0878ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.858325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd10f9-871f-40c8-b751-6e339b0878ea admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.868798 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1442/2852] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 19:27:03.872776 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:03.882756 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2deaf-c8a5-4eae-8733-f67f203b12f1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:03.887420 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:03.888058 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.888334 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.888608 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] PATH_INFO: `/v3/projects/ef884f74727e4b1bb887529836b17ba1/users/6e36f8558db54b59b7c076203687eba1/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:03.903559 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef884f74727e4b1bb887529836b17ba1, user_id=6e36f8558db54b59b7c076203687eba1, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:03.906732 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:03.970347 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:03.974861 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1411/2853] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:27:03 2022] PUT /identity/v3/projects/ef884f74727e4b1bb887529836b17ba1/users/6e36f8558db54b59b7c076203687eba1/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:03.985204 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:03.985491 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:03.985797 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:04.013239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] MFA Rules not processed for user `6e36f8558db54b59b7c076203687eba1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:04.014448 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:04.070874 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:04.081530 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1443/2854] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:04.092577 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:04.094247 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:04.150455 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f666a-9b08-4dec-bddf-e94eee829060 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:04.154635 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85356-7d5d-4cbb-b311-26e230448bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:04.155241 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85356-7d5d-4cbb-b311-26e230448bdd service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:04.155523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85356-7d5d-4cbb-b311-26e230448bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:04.155841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85356-7d5d-4cbb-b311-26e230448bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:04.164075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e85356-7d5d-4cbb-b311-26e230448bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:04.165534 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e85356-7d5d-4cbb-b311-26e230448bdd service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:04.174659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1412/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:04.193775 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:04.256779 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9388f-9466-480c-b053-0b0fc280ba55 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:04.261533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:04.262161 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:04.262439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:04.262743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] PATH_INFO: `/v3/users/6e36f8558db54b59b7c076203687eba1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:04.263866 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e36f8558db54b59b7c076203687eba1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:04.266455 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:04.410653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e36f8558db54b59b7c076203687eba1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:04.411355 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e36f8558db54b59b7c076203687eba1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:04.411994 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e36f8558db54b59b7c076203687eba1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:04.419016 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e36f8558db54b59b7c076203687eba1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:04.431778 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e36f8558db54b59b7c076203687eba1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:04.516840 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1444/2856] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:27:04 2022] DELETE /identity/v3/users/6e36f8558db54b59b7c076203687eba1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:04.521571 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:04.521848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:04.522120 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:04.542283 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:04.543942 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:04.601908 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1413/2857] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:04.612138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:04.613529 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:04.671130 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88f65-6d99-44c9-89cd-9253f5263ad0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:04.674951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d952b8-bf42-4a90-8db0-86743f7f7f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:04.675548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d952b8-bf42-4a90-8db0-86743f7f7f05 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:04.675829 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d952b8-bf42-4a90-8db0-86743f7f7f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:04.676104 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d952b8-bf42-4a90-8db0-86743f7f7f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:04.685028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d952b8-bf42-4a90-8db0-86743f7f7f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:04.686438 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d952b8-bf42-4a90-8db0-86743f7f7f05 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:04.695891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1445/2858] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:05.118046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.119413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:05.131480 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258616ba-7f3a-4022-ba7c-12f334a0d491 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.136900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.137454 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.137632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.137995 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] PATH_INFO: `/v3/projects/ef884f74727e4b1bb887529836b17ba1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.139141 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef884f74727e4b1bb887529836b17ba1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.144459 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.254137 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef884f74727e4b1bb887529836b17ba1', 'request_id': 'req-cf765008-4156-4ff9-aebe-2dfff6af6b99'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:05.260359 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef884f74727e4b1bb887529836b17ba1', 'request_id': 'req-cf765008-4156-4ff9-aebe-2dfff6af6b99'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:05.266400 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef884f74727e4b1bb887529836b17ba1', 'request_id': 'req-cf765008-4156-4ff9-aebe-2dfff6af6b99'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:05.345210 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] The token cache is being invalidate because project ef884f74727e4b1bb887529836b17ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:27:05.345556 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:05.346191 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cf765008-4156-4ff9-aebe-2dfff6af6b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:05.347726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1414/2859] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:27:05 2022] DELETE /identity/v3/projects/ef884f74727e4b1bb887529836b17ba1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:05.352662 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.353069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.353486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.373128 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:05.374383 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:05.384014 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:05.435411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1446/2860] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:05.443060 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.443370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.445584 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.463751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:05.465135 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:05.473541 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1415/2861] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:05.477718 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.487512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfb415-37a6-4f35-913f-0f598dc47a62 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.492154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.492750 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.493026 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.493297 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.493944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:27:05.494243 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.495868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.507900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1447/2862] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:27:05.512349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.522119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa61147-6bdb-4f4e-8693-1b961b141882 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.527046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.527661 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.527935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.528198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.528846 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.530751 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.602701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:05.609468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1416/2863] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:27:05 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:27:05.613823 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.623900 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f673f-2f9f-4f95-aa1b-950aaaa3b7fc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.680986 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.681595 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.681938 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.682242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.682929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.684821 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.744786 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1448/2864] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:27:05 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:27:05.748813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.760230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d1d4b-8485-4fb2-a51b-6e05708b14da admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.764906 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.765499 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.765778 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.766709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.767382 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.768966 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.779046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1417/2865] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:05.782814 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.792794 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066165b-d223-4af1-be06-1f3d06263092 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.797729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce375a2-f8dc-4939-8448-fe19467af669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.798420 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce375a2-f8dc-4939-8448-fe19467af669 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.798752 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce375a2-f8dc-4939-8448-fe19467af669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.799030 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce375a2-f8dc-4939-8448-fe19467af669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.799660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce375a2-f8dc-4939-8448-fe19467af669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.801221 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce375a2-f8dc-4939-8448-fe19467af669 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.811629 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1449/2866] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:05.816179 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:05.825919 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aa30c-f31d-45c4-a5f4-607531c5c74e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:05.831172 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:05.831804 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.832081 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.832375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] PATH_INFO: `/v3/projects/5f8565bab398420cb670f60c14663227/users/f1e144645fb149b686abed6668e7a9d2/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:05.846717 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8565bab398420cb670f60c14663227, user_id=f1e144645fb149b686abed6668e7a9d2, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:05.849330 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:05.910797 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1418/2867] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:27:05 2022] PUT /identity/v3/projects/5f8565bab398420cb670f60c14663227/users/f1e144645fb149b686abed6668e7a9d2/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:05.973811 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:05.974122 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:05.974401 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:06.002984 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] MFA Rules not processed for user `f1e144645fb149b686abed6668e7a9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:06.004195 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:06.066865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1450/2868] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:06.077962 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:06.079748 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:06.136025 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560f5e-082c-42ba-a13a-05627f757284 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:06.139727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81166d-02c5-402b-a23d-1ccd71ac339b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:06.140354 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81166d-02c5-402b-a23d-1ccd71ac339b service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:06.140630 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81166d-02c5-402b-a23d-1ccd71ac339b service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:06.140973 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81166d-02c5-402b-a23d-1ccd71ac339b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:06.149393 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81166d-02c5-402b-a23d-1ccd71ac339b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:06.151088 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81166d-02c5-402b-a23d-1ccd71ac339b service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:06.159902 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1419/2869] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:15.909831 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:15.921879 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:15.971633 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b476c-25ab-4346-8a5f-b2bc77514b7c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:15.976192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:15.977599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:15.977599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:15.977599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] PATH_INFO: `/v3/users/f1e144645fb149b686abed6668e7a9d2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:15.978616 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e144645fb149b686abed6668e7a9d2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:15.980519 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:16.153671 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e144645fb149b686abed6668e7a9d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.154325 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e144645fb149b686abed6668e7a9d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.154936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e144645fb149b686abed6668e7a9d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.155783 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:16.162065 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e144645fb149b686abed6668e7a9d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.174033 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e144645fb149b686abed6668e7a9d2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.254007 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1451/2870] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:27:15 2022] DELETE /identity/v3/users/f1e144645fb149b686abed6668e7a9d2 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:16.258476 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:16.259023 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:16.259568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:16.279443 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:16.280681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:16.338429 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1420/2871] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:16.348121 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:16.349515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:16.406421 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229bdf1-a46b-475f-bcff-51a834b94e2e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:16.409956 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2334-adc6-4ca1-bb5c-5aff352fd3d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:16.410581 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2334-adc6-4ca1-bb5c-5aff352fd3d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:16.410869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2334-adc6-4ca1-bb5c-5aff352fd3d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:16.411141 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2334-adc6-4ca1-bb5c-5aff352fd3d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:16.420070 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad2334-adc6-4ca1-bb5c-5aff352fd3d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:16.421489 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad2334-adc6-4ca1-bb5c-5aff352fd3d0 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:16.431368 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1452/2872] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:16.826821 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:16.828238 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:16.843007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51ced0-2f6e-44d1-863f-166a2deacdf0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:16.848345 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:16.848977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:16.849257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:16.849529 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] PATH_INFO: `/v3/projects/5f8565bab398420cb670f60c14663227` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:16.850849 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8565bab398420cb670f60c14663227)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:16.857403 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:16.945945 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8565bab398420cb670f60c14663227', 'request_id': 'req-a023fb3e-b839-452c-a158-33c021912a89'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.952157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8565bab398420cb670f60c14663227', 'request_id': 'req-a023fb3e-b839-452c-a158-33c021912a89'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:16.957947 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8565bab398420cb670f60c14663227', 'request_id': 'req-a023fb3e-b839-452c-a158-33c021912a89'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:17.012625 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] The token cache is being invalidate because project 5f8565bab398420cb670f60c14663227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:27:17.012909 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:17.013509 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a023fb3e-b839-452c-a158-33c021912a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:17.015052 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1421/2873] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:27:16 2022] DELETE /identity/v3/projects/5f8565bab398420cb670f60c14663227 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:17.019915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.022076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.022076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.040427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:17.041615 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:17.071237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:17.101165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1453/2874] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:17.108999 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.109315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.109597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.130014 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:17.131375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:17.139409 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1422/2875] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:17.144187 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.154053 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2c2fa-7861-4473-b88e-8e9a3f3cf64d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.159291 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.159882 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.160160 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.160431 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.161069 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:27:17.161369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.162978 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.174746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1454/2876] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:27:17.179050 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.188376 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512590a-9bd6-4eb9-b05d-5743f2aa2a40 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.192991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.193582 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.193862 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.194133 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.194880 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.196841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.294939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1423/2877] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:27:17 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:27:17.299260 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.309515 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137673c-2751-427b-b9d8-1cf97664a967 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.365344 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.365945 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.366226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.366501 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.367171 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.369066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.447884 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1455/2878] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:27:17 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 19:27:17.451867 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.462753 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95aaba-14ab-4cd6-9f8d-edbc76babcdc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.467405 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.470364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.470364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.470364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.470364 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.471500 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.481894 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1424/2879] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:17.485852 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.496060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026eb4-ecff-478f-a334-4928efd6d3c0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.501666 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bf9b8-a8cc-4d19-8f74-c97a015aad24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.502262 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bf9b8-a8cc-4d19-8f74-c97a015aad24 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.502539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bf9b8-a8cc-4d19-8f74-c97a015aad24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.502843 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bf9b8-a8cc-4d19-8f74-c97a015aad24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.503486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bf9b8-a8cc-4d19-8f74-c97a015aad24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.505173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bf9b8-a8cc-4d19-8f74-c97a015aad24 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.515889 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1456/2880] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:17.520645 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.531230 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b784346-6b0e-44d8-8a81-b31608f99ca1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.536067 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.536703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.536978 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.537253 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] PATH_INFO: `/v3/projects/3c1b2d9678d74e7c9c907209045deb3d/users/dc1a09dd15bb4882a533331bf507f46e/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.553122 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1b2d9678d74e7c9c907209045deb3d, user_id=dc1a09dd15bb4882a533331bf507f46e, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.555915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.608024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1425/2881] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:27:17 2022] PUT /identity/v3/projects/3c1b2d9678d74e7c9c907209045deb3d/users/dc1a09dd15bb4882a533331bf507f46e/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:17.618535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.618845 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.619154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.649675 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] MFA Rules not processed for user `dc1a09dd15bb4882a533331bf507f46e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:17.650944 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:17.715404 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1457/2882] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:17.726568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.728286 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:17.749567 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:17.785438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d246713-1c8c-4b0b-8f4a-3f62d63d797e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.789065 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e2fb8-bb99-4c8d-968c-619ab9a0fed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.789668 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e2fb8-bb99-4c8d-968c-619ab9a0fed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.789943 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e2fb8-bb99-4c8d-968c-619ab9a0fed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.790215 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e2fb8-bb99-4c8d-968c-619ab9a0fed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.798352 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e2fb8-bb99-4c8d-968c-619ab9a0fed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.799824 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e2fb8-bb99-4c8d-968c-619ab9a0fed3 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:17.808703 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1426/2883] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:17.833738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:17.863880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:17.896954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbce251-af4f-4fef-a508-9600b57ae80e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:17.903261 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:17.904186 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:17.906638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:17.906638 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] PATH_INFO: `/v3/users/dc1a09dd15bb4882a533331bf507f46e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:17.906993 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1a09dd15bb4882a533331bf507f46e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:17.908910 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:18.106702 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1a09dd15bb4882a533331bf507f46e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.111293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1a09dd15bb4882a533331bf507f46e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.111293 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1a09dd15bb4882a533331bf507f46e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.115609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1a09dd15bb4882a533331bf507f46e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.128067 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1a09dd15bb4882a533331bf507f46e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.179125 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1458/2884] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:27:17 2022] DELETE /identity/v3/users/dc1a09dd15bb4882a533331bf507f46e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:18.183775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:18.185882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:18.185882 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:18.203814 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:18.204659 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:18.270816 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1427/2885] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:18.286604 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:18.288109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:18.345535 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960282a5-ed40-42d9-bbd0-b4327dcbf323 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:18.349269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4de92f-8e22-4a45-b515-543a308522dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:18.349893 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4de92f-8e22-4a45-b515-543a308522dc service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:18.350190 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4de92f-8e22-4a45-b515-543a308522dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:18.350460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4de92f-8e22-4a45-b515-543a308522dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:18.359324 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4de92f-8e22-4a45-b515-543a308522dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:18.361314 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4de92f-8e22-4a45-b515-543a308522dc service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:18.370988 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1459/2886] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:18.769506 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:18.771901 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:18.783568 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73a4cd-5741-4798-92a5-f348f5673dd8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:18.788199 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:18.788833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:18.789094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:18.789542 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] PATH_INFO: `/v3/projects/3c1b2d9678d74e7c9c907209045deb3d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:18.790769 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1b2d9678d74e7c9c907209045deb3d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:18.792688 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:18.897765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1b2d9678d74e7c9c907209045deb3d', 'request_id': 'req-049a2052-90a4-42dc-b060-fc7c4ba370fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.904041 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1b2d9678d74e7c9c907209045deb3d', 'request_id': 'req-049a2052-90a4-42dc-b060-fc7c4ba370fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.910264 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1b2d9678d74e7c9c907209045deb3d', 'request_id': 'req-049a2052-90a4-42dc-b060-fc7c4ba370fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.961492 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] The token cache is being invalidate because project 3c1b2d9678d74e7c9c907209045deb3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:27:18.961775 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.962402 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-049a2052-90a4-42dc-b060-fc7c4ba370fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:27:18.963977 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1428/2887] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:27:18 2022] DELETE /identity/v3/projects/3c1b2d9678d74e7c9c907209045deb3d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:18.973873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:18.973873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:18.973873 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:18.990865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:18.992066 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:19.039799 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:19.054539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1460/2888] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:27:19.063935 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.064220 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.064496 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.084326 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:19.085532 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:19.093694 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1429/2889] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:19.098773 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.108635 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04de6-9b2d-4049-9281-a60c8b1cbf9f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.113743 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.114321 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.114642 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.114915 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.115557 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:27:19.115855 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.117418 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.129034 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1461/2890] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 19:27:19.133115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.142223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8bb79-83a6-498d-a9bf-dff2f2c50e18 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.146938 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.147523 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.147853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.148075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.148708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.150623 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.238848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1430/2891] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:27:19 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:27:19.243096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.253158 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb7662-1c60-44bc-acd3-d4fe3cb1ab0f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.309911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.310512 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.310819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.311088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.311724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.314151 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.419184 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1462/2892] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:27:19 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:27:19.423833 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.434104 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a43fa-4fa4-4f94-9295-11386eaa36d9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.439495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.440113 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.440397 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.440669 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.441300 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.442900 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.452562 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1431/2893] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:19.456679 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.466444 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc8301-b644-4909-b48e-df59d5d6f9f5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.470972 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231f915-a3b4-45d0-ad29-7db14bcd0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.471556 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231f915-a3b4-45d0-ad29-7db14bcd0d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.471830 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231f915-a3b4-45d0-ad29-7db14bcd0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.472097 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231f915-a3b4-45d0-ad29-7db14bcd0d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.472727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231f915-a3b4-45d0-ad29-7db14bcd0d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.474307 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231f915-a3b4-45d0-ad29-7db14bcd0d09 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.485558 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1463/2894] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:27:19.489528 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.499487 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45269-2099-4673-bd93-70f5e72c3dae admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.504727 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.505362 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.505664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.505939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] PATH_INFO: `/v3/projects/8e25fa2f514648aa80ca55da8bb48cef/users/05d4c61badff44108fb7f2ed329b50c8/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.508167 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:19.520681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e25fa2f514648aa80ca55da8bb48cef, user_id=05d4c61badff44108fb7f2ed329b50c8, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.523322 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.596257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1432/2895] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:27:19 2022] PUT /identity/v3/projects/8e25fa2f514648aa80ca55da8bb48cef/users/05d4c61badff44108fb7f2ed329b50c8/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:27:19.601285 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8d57-0a51-4bda-a452-b0cb6667c7b5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.601572 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8d57-0a51-4bda-a452-b0cb6667c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.601848 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8d57-0a51-4bda-a452-b0cb6667c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.620932 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b66a8d57-0a51-4bda-a452-b0cb6667c7b5 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:19.622129 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b66a8d57-0a51-4bda-a452-b0cb6667c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:19.681788 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1464/2896] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:27:19.691341 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:19.692993 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:19.748452 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac860eb8-42cf-4723-860d-6d445c56c503 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:19.752597 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:19.753188 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:19.753468 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:19.753740 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:19.762653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:19.764082 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:19.774059 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1433/2897] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:20.900411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68025ead-9553-44e8-b063-f2eec8a72c67 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:20.901774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68025ead-9553-44e8-b063-f2eec8a72c67 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:20.901774 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68025ead-9553-44e8-b063-f2eec8a72c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:20.929738 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-68025ead-9553-44e8-b063-f2eec8a72c67 None None] MFA Rules not processed for user `05d4c61badff44108fb7f2ed329b50c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:20.931000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-68025ead-9553-44e8-b063-f2eec8a72c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:20.977192 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:20.997310 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1465/2898] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:27:21.008914 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:21.010039 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:21.068160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d98ed-6825-45e6-a38c-6b7db100109b service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:21.071701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:21.072305 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:21.072583 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:21.072855 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:21.081044 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:21.082469 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:21.091228 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1434/2899] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:22.773364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:22.773912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:22.773912 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:22.793427 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] MFA Rules not processed for user `05d4c61badff44108fb7f2ed329b50c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:27:22.794660 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:22.801991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1466/2900] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:27:23.035904 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:23.045418 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368af1-7b4b-49e2-ad17-11fd8e1a6037 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:23.049681 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:23.050271 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:23.050564 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:23.050848 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:23.052579 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:23.062160 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:23.063619 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a61d6d-f953-4968-815d-428547bfb367 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:23.072390 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1435/2901] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:32.788012 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=9 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:33.068174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:27:47.967063 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:27:47.968478 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:48.039385 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb047d95-1f6a-44ec-a907-cb2529e448fa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:27:48.044975 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:27:48.045583 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:27:48.045897 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:27:48.046181 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:27:48.047948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:27:48.059960 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:27:48.061526 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e587b2-3cab-4355-8da7-d4f3a28cd3f2 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:27:48.071486 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1467/2902] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:27:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:27:58.067165 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=32 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:28:30.563195 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5eea4-74f9-4b84-bd5f-3527caca4385 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:30.563759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5eea4-74f9-4b84-bd5f-3527caca4385 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:30.563759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5eea4-74f9-4b84-bd5f-3527caca4385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:30.583457 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9bc5eea4-74f9-4b84-bd5f-3527caca4385 None None] MFA Rules not processed for user `05d4c61badff44108fb7f2ed329b50c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:28:30.584646 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9bc5eea4-74f9-4b84-bd5f-3527caca4385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:28:30.591760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1436/2903] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:28:34.540948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae987d9-8536-4862-b3fd-998efe4e787a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:34.541545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae987d9-8536-4862-b3fd-998efe4e787a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:34.541545 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae987d9-8536-4862-b3fd-998efe4e787a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:34.543269 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1468/2904] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:28:34.549853 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d8a2-0ba2-466e-b2a0-cb3e721a7c6f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:34.550438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d8a2-0ba2-466e-b2a0-cb3e721a7c6f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:34.550438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d8a2-0ba2-466e-b2a0-cb3e721a7c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:34.594834 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6d91d8a2-0ba2-466e-b2a0-cb3e721a7c6f None None] MFA Rules not processed for user `4267b9d353724a9fab912e6bd0c2ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:28:34.596064 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6d91d8a2-0ba2-466e-b2a0-cb3e721a7c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:28:34.604315 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1437/2905] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:28:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:28:34.625050 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae987d9-8536-4862-b3fd-998efe4e787a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:28:34.637612 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae987d9-8536-4862-b3fd-998efe4e787a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:28:34.641548 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c95297-1520-4c65-9ba3-eb097d894101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:28:34.642173 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95297-1520-4c65-9ba3-eb097d894101 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:34.642451 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95297-1520-4c65-9ba3-eb097d894101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:34.642763 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95297-1520-4c65-9ba3-eb097d894101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:34.651237 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95297-1520-4c65-9ba3-eb097d894101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:28:34.653819 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95297-1520-4c65-9ba3-eb097d894101 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:28:34.663430 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1469/2906] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:28:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 19:28:35.048708 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81212e5b-7e24-4436-b1b7-ada9346aff58 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:35.048996 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81212e5b-7e24-4436-b1b7-ada9346aff58 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:35.049276 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81212e5b-7e24-4436-b1b7-ada9346aff58 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:35.050706 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1438/2907] 10.222.0.46 () {54 vars in 921 bytes} [Wed Sep 7 19:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 19:28:35.055499 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:35.055954 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:35.056226 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:35.095669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] MFA Rules not processed for user `a500e08c87dc4e45899d7408e7b906cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:28:35.097002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:28:35.178933 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1470/2908] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Sep 7 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:28:35.189820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81212e5b-7e24-4436-b1b7-ada9346aff58 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:28:35.201594 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81212e5b-7e24-4436-b1b7-ada9346aff58 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:28:35.205180 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:28:35.205980 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:28:35.206262 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:28:35.206540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:28:35.221599 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:28:35.223089 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:28:35.234154 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1439/2909] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:28:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:28:45.169514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=43 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:28:45.229245 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=18 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:29:38.935692 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:38.946330 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7adef5-e881-41a9-8f22-df085d73a0cb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:38.951076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:38.951687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:38.951966 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:38.952239 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] PATH_INFO: `/v3/users/05d4c61badff44108fb7f2ed329b50c8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:38.953369 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d4c61badff44108fb7f2ed329b50c8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:38.955303 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:39.110242 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d4c61badff44108fb7f2ed329b50c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.110888 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d4c61badff44108fb7f2ed329b50c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.111507 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d4c61badff44108fb7f2ed329b50c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.118653 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d4c61badff44108fb7f2ed329b50c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.130651 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3661993f-5e81-4cdc-8d8a-a5ce9571511e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d4c61badff44108fb7f2ed329b50c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.208796 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1471/2910] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:29:38 2022] DELETE /identity/v3/users/05d4c61badff44108fb7f2ed329b50c8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:29:39.434419 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:39.445090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0998-75b9-4a9e-84ba-4440ccda115a service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:39.503175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:39.503803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:39.504079 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:39.504360 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] PATH_INFO: `/v3/projects/8e25fa2f514648aa80ca55da8bb48cef` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:39.505470 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e25fa2f514648aa80ca55da8bb48cef)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:39.507438 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:39.597884 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e25fa2f514648aa80ca55da8bb48cef', 'request_id': 'req-50dd3122-2274-4eab-93bf-dfb9c37360e0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.604119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e25fa2f514648aa80ca55da8bb48cef', 'request_id': 'req-50dd3122-2274-4eab-93bf-dfb9c37360e0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.609696 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e25fa2f514648aa80ca55da8bb48cef', 'request_id': 'req-50dd3122-2274-4eab-93bf-dfb9c37360e0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.671851 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] The token cache is being invalidate because project 8e25fa2f514648aa80ca55da8bb48cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:29:39.672196 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.672822 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-50dd3122-2274-4eab-93bf-dfb9c37360e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:39.674289 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1440/2911] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:29:39 2022] DELETE /identity/v3/projects/8e25fa2f514648aa80ca55da8bb48cef => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:29:39.679392 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:39.679606 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:39.679880 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:39.701462 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:29:39.702693 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:39.763098 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1472/2912] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:29:39.771047 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:39.771333 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:39.771634 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:39.791189 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:29:39.792391 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:39.800198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1441/2913] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 19:29:39.804120 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:39.814154 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aea8e-f4d4-4023-9ad1-befd8e7a9f2b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:39.818847 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:39.819439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:39.819719 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:39.819997 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:39.820637 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 19:29:39.820936 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:39.823175 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:39.834609 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1473/2914] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Sep 7 19:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 19:29:39.838709 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:39.848157 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2845-3732-486f-878f-6fc3c9409b32 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:39.852841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:39.853465 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:39.853745 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:39.854016 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:39.854726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:39.856620 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:39.936760 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1442/2915] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Sep 7 19:29:39 2022] POST /identity/v3/projects => generated 397 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:29:39.940837 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:39.959727 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6085a-cd86-488c-88b5-d17a639b9488 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:40.007787 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:40.008386 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:40.008668 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:40.008970 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:40.009599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:40.012091 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:40.019977 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:29:40.107728 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1474/2916] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Sep 7 19:29:39 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 19:29:40.110831 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:40.121820 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9dd23-fba3-42ba-81cd-ddd14655f40f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:40.126442 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:40.127062 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:40.127339 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:40.127610 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:40.128294 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:40.129885 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:40.139726 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1443/2917] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:29:40.143929 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:40.155079 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74850f8f-ffec-4a56-b648-0bde7e79472a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:40.159946 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879073-d89b-49d3-9191-d4841f762f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:40.160541 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879073-d89b-49d3-9191-d4841f762f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:40.160822 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879073-d89b-49d3-9191-d4841f762f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:40.161096 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879073-d89b-49d3-9191-d4841f762f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:40.161735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879073-d89b-49d3-9191-d4841f762f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:40.163336 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879073-d89b-49d3-9191-d4841f762f69 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:40.174352 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1475/2918] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Sep 7 19:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 19:29:40.178198 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:40.188632 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f034a6-9156-4d74-b6df-2eb62c0709d1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:40.193158 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:40.193793 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:40.194072 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:40.194349 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] PATH_INFO: `/v3/projects/4ead53b550bc40439a89eb059c0deff3/users/6da98f24ffde4bc1a34b900284b6007b/roles/8a09ae9946bc4e158d491383af34a89f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:40.208626 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ead53b550bc40439a89eb059c0deff3, user_id=6da98f24ffde4bc1a34b900284b6007b, role_id=8a09ae9946bc4e158d491383af34a89f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:40.211306 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:40.290007 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1444/2919] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Sep 7 19:29:40 2022] PUT /identity/v3/projects/4ead53b550bc40439a89eb059c0deff3/users/6da98f24ffde4bc1a34b900284b6007b/roles/8a09ae9946bc4e158d491383af34a89f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:29:40.300990 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:40.305868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:40.305868 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:40.330414 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] MFA Rules not processed for user `6da98f24ffde4bc1a34b900284b6007b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:29:40.331657 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:40.395410 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1476/2920] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:29:40.407078 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:40.408759 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:40.439841 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:29:40.466472 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a098c5-2d6f-41f9-aa50-61bcd9559ecc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:40.470175 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205df2ea-731a-4aba-bd94-d0fa666b5817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885f09bf588a442e931aae0da51d0ee2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:40.470813 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205df2ea-731a-4aba-bd94-d0fa666b5817 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:40.471094 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205df2ea-731a-4aba-bd94-d0fa666b5817 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:40.471426 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205df2ea-731a-4aba-bd94-d0fa666b5817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:40.482733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205df2ea-731a-4aba-bd94-d0fa666b5817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:40.484173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205df2ea-731a-4aba-bd94-d0fa666b5817 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:40.493102 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1445/2921] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:29:49.462278 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:49.526411 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec171f4d-eb49-4bfb-be8a-23f55ac3c75d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:49.531004 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:49.531626 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:49.531906 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:49.532183 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] PATH_INFO: `/v3/users/6da98f24ffde4bc1a34b900284b6007b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:49.533333 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da98f24ffde4bc1a34b900284b6007b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:49.535281 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:49.673687 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da98f24ffde4bc1a34b900284b6007b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:49.674331 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da98f24ffde4bc1a34b900284b6007b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:49.674948 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da98f24ffde4bc1a34b900284b6007b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:49.682646 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da98f24ffde4bc1a34b900284b6007b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:49.692782 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:29:49.694927 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da98f24ffde4bc1a34b900284b6007b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:49.740022 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1477/2922] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:29:49 2022] DELETE /identity/v3/users/6da98f24ffde4bc1a34b900284b6007b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:29:49.750270 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:49.750819 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:49.750936 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:49.770701 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] MFA Rules not processed for user `bc4a17a3aa754cf9b61c2b367071735b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 19:29:49.771929 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:49.829951 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1446/2923] 10.222.0.46 () {58 vars in 958 bytes} [Wed Sep 7 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 19:29:49.839706 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:49.841099 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:49.897332 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e36884-2b89-4baf-9817-7bc8b86aa88b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:49.901274 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:49.901869 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:49.902147 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:49.902439 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:49.911320 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:49.912729 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:49.922647 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1478/2924] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:29:50.307119 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:29:50.308656 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:29:50.321736 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e76a21-be24-44f6-bd24-679975d04a6b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:29:50.326697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:29:50.328379 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:29:50.328664 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:29:50.328991 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] PATH_INFO: `/v3/projects/4ead53b550bc40439a89eb059c0deff3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:29:50.330149 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ead53b550bc40439a89eb059c0deff3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:29:50.332115 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:29:50.442603 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ead53b550bc40439a89eb059c0deff3', 'request_id': 'req-cc6d0065-ed41-4108-a5a4-0c396deec2d5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:50.448806 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ead53b550bc40439a89eb059c0deff3', 'request_id': 'req-cc6d0065-ed41-4108-a5a4-0c396deec2d5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:50.454435 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ead53b550bc40439a89eb059c0deff3', 'request_id': 'req-cc6d0065-ed41-4108-a5a4-0c396deec2d5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:50.533046 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] The token cache is being invalidate because project 4ead53b550bc40439a89eb059c0deff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:29:50.533367 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:50.533984 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:29:50.535515 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1447/2925] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:29:50 2022] DELETE /identity/v3/projects/4ead53b550bc40439a89eb059c0deff3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:29:59.918446 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=2,keystone:SELECT=29,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:30:00.530051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:30:12.750539 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:12.759828 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d5982-b956-40c7-976c-7726642c90ee service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:12.811674 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:12.812287 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:12.812566 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:12.812959 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:12.814684 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:12.910985 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:12.912722 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:12.926724 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1479/2926] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:30:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:30:14.132495 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:14.133917 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:14.146291 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d0065-ed41-4108-a5a4-0c396deec2d5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:14.150146 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:14.150802 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:14.151093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:14.151370 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:14.153051 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:14.167279 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:14.168693 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:14.180257 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1448/2927] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:30:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:30:14.955027 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:14.956465 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:14.970525 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc4db1-9729-45b9-900c-634b26d2045c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:14.976076 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a500e08c87dc4e45899d7408e7b906cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:14.976670 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:14.976951 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:14.977228 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:14.979002 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:14.991138 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:14.995460 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:15.004231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1480/2928] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:30:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:30:23.939869 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:23.950653 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ca5c-230e-4d35-9a4b-0554a23b6680 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:23.954925 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4267b9d353724a9fab912e6bd0c2ab2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:23.955527 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:23.955803 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:23.956074 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:23.957771 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:24.021790 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:24.023424 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:24.033768 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1449/2929] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:30:24.999514 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=60 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:30:26.277509 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:26.279028 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:26.293536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b6b7c-625c-46fb-b53c-d49bd029123c service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:26.299109 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:26.299735 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:26.300010 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:26.300282 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] PATH_INFO: `/v3/users/5076bee4033c4fa8837c3273a2e53a7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:26.301388 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5076bee4033c4fa8837c3273a2e53a7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:26.303381 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:26.462364 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5076bee4033c4fa8837c3273a2e53a7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:26.463041 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5076bee4033c4fa8837c3273a2e53a7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:26.463696 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5076bee4033c4fa8837c3273a2e53a7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:26.471422 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5076bee4033c4fa8837c3273a2e53a7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:26.483536 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5076bee4033c4fa8837c3273a2e53a7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:26.557305 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1481/2930] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:30:26 2022] DELETE /identity/v3/users/5076bee4033c4fa8837c3273a2e53a7a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:30:26.787765 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:26.798093 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f30d7-ad2e-4526-b845-674e612bb154 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:26.856733 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:26.857358 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:26.857638 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:26.858024 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] PATH_INFO: `/v3/projects/67ebd705b0d94555bbc24018ac4b9d0f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:26.859174 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ebd705b0d94555bbc24018ac4b9d0f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:26.861075 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:27.008377 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ebd705b0d94555bbc24018ac4b9d0f', 'request_id': 'req-c12de79b-96a4-459c-95ae-2922a30ecd2c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.014540 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ebd705b0d94555bbc24018ac4b9d0f', 'request_id': 'req-c12de79b-96a4-459c-95ae-2922a30ecd2c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.020114 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ebd705b0d94555bbc24018ac4b9d0f', 'request_id': 'req-c12de79b-96a4-459c-95ae-2922a30ecd2c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.091817 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] The token cache is being invalidate because project 67ebd705b0d94555bbc24018ac4b9d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:30:27.092109 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.092770 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.094857 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1450/2931] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:30:26 2022] DELETE /identity/v3/projects/67ebd705b0d94555bbc24018ac4b9d0f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 19:30:27.098911 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:27.100358 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:27.114020 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a12c1-abbc-4356-b581-99b00105e936 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:27.179865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:27.180533 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:27.180812 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:27.181088 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] PATH_INFO: `/v3/users/fbb00d3e463e45e586b274af0de1f4d5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:27.182231 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb00d3e463e45e586b274af0de1f4d5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:27.184182 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:27.370060 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb00d3e463e45e586b274af0de1f4d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.370723 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb00d3e463e45e586b274af0de1f4d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.371263 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb00d3e463e45e586b274af0de1f4d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.378669 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb00d3e463e45e586b274af0de1f4d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.392100 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb00d3e463e45e586b274af0de1f4d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.393746 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:30:27.476363 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1482/2932] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Sep 7 19:30:27 2022] DELETE /identity/v3/users/fbb00d3e463e45e586b274af0de1f4d5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:30:27.730697 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:30:27.732674 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:30:27.746090 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de79b-96a4-459c-95ae-2922a30ecd2c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:30:27.754939 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:30:27.806056 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4a17a3aa754cf9b61c2b367071735b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eeffadf972f483fb4e32d2c25b5e94c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:30:27.806826 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:30:27.806988 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:30:27.807261 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] PATH_INFO: `/v3/projects/a43589ded36843fd8bb9c7046d9614df` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:30:27.808375 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43589ded36843fd8bb9c7046d9614df)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:30:27.810413 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:30:27.922915 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43589ded36843fd8bb9c7046d9614df', 'request_id': 'req-f2f5a314-53ff-4ca0-9c65-194bc81cc975'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.929223 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43589ded36843fd8bb9c7046d9614df', 'request_id': 'req-f2f5a314-53ff-4ca0-9c65-194bc81cc975'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.934897 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43589ded36843fd8bb9c7046d9614df', 'request_id': 'req-f2f5a314-53ff-4ca0-9c65-194bc81cc975'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.990295 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] The token cache is being invalidate because project a43589ded36843fd8bb9c7046d9614df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 19:30:27.990518 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.991186 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f2f5a314-53ff-4ca0-9c65-194bc81cc975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 19:30:27.992713 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1451/2933] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Sep 7 19:30:27 2022] DELETE /identity/v3/projects/a43589ded36843fd8bb9c7046d9614df => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 19:30:37.546000 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:30:37.987173 dv-cinder-847732-5-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 19:32:51.870865 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 19:32:51.872339 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:32:52.017511 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2a696-3bbb-40ea-ac42-6fd5a3a339a1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 19:32:52.079991 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c32b1292d0c6436ba4669dbfd1c73ea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0384adf02c8a481aaa2816ff8339e704', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 19:32:52.080599 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 19:32:52.080877 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 19:32:52.081150 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 19:32:52.082891 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 19:32:52.094627 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 19:32:52.096256 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe17b-c7c4-4826-81c7-2e7e8247f5cd service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 19:32:52.106351 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1483/2934] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Sep 7 19:32:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 19:33:02.101863 dv-cinder-847732-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}